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(54) Video display system with picture-in-picture facility 

(57) A video processor has two inputs, FULL SCREEN and PIP. The processor decodes the second input PIP 82 
and scales it 83 to produce a reduced digital video signal which is passed to a video encoder 86. Synchronising 
circuits 84,85 generate synchronising information which is used by the video encoder 86 to produce a reduced 
analogue signal. This video signal is then combined 88 with the first input signal FULL SCREEN to produce a 
combined signal for display. 
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VIDEO DISPLAY SYSTEM 
Background of the Invention 

This invention relates to consumer use of what is here called the 
"television space". That is, the use of video/audio signal streams such 
as have been in the past distributed by broadcast over radio frequency 
bands or by cable distribution, or made available from video 
recorder/player devices such as cassette recorders or video disc players, 
or made available from direct, live sources such as cameras, game systems 
or computers. Such video/audio signal streams, whether carrying analog 
or digitally encoded information, have come to represent a significant 
resource to most consumers for information and entertainment. 

Access to the television space has, in the past, been achieved by 
use of a television receiver. Then came changes in the methods of 
distribution, leading to the use of various set top devices such as cable 
boxes for analog signal streams, recorder/players, game machines, home 
cameras, etc. As television space technology has approached what is 
presently known as the -home theatre", systems having as many as seven or 
more constituent components which are connected one to another have 
become possible. in such a system of systems, several or even all of the 
constituent systems may have its own remote control device, intended to 
enable a human observer to control the functionality of the respective 
constituent system while avoiding the necessity of directly manipulating 
control available at the face of the system. with the proliferation of 
systems, a user is frequently faced with a proliferation of remote 
control devices . 

in addition, as the above mentioned devices and components using 
the television space have proliferated, so also have the multitude of 
problems that must be solved when connecting such devices and components 
such as, for example, TV's, video cassette recorder/player (s) (vcr(s)) 
devices, cable boxes, cameras, game systems, etc. These problems include 
difficulty in programming the VCR, multiple remote controls and 
associated hardware connected to the TV including complex interconnect 
cabling between the components. 

The present invention proposes that these difficulties be resolved 
by providing a highly integrated common memory video processor for 
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managing live video images, graphics, audio and compact disc data flow 
necessary in a set top device which is coupled to multiple devices and 
components. Further, a single memory interface for system, video, 
graphics and electronic program guide (EPG) information is provided. in 
addition, a single remote control device is provided which has access to 
the resources of the entire system with which it is related and has 
minimal buttons to be actuated by the human observer. 

Disclosure of the Invention 

With the above discussion in mind, it is one purpose of this 
invention to assist a human observer of programming made available in the 
television space, or similar displays found elsewhere, in making 
selections of services or functions to be accessed through the system 
displaying the visual images so derived. In accomplishing this purpose, 
the present invention is directed to a method and apparatus for receiving 
first and second analog composite video signals and producing an output 
analog video signal comprised of a portion of the first analog composite 
video signal and a portion of the second analog composite video signal. 
More specifically, the present invention is directed to a video processor 
which includes a video decoder and scaler for converting the second 
analog composite video signal to a reduced digital video signal. The 
video processor also includes a video encoder for converting the reduced 
digital video to a third analog composite video signal. Synchronization 
circuits are included for producing synchronization information for 
synchronizing the third analog composite video signal with the first 
analog composite video signal. The video processor further includes a 
composite video multiplexer for combining the first analog composite 
video signal with the third analog composite signal to produce the output 
analog composite video signal. 

Brief Description of the Drawings 

The invention will now be described, by way of example only, with 
reference to the accompanying drawings, in which: 

Figure 1 is a perspective view of one embodiment of the present 
invention which includes a television receiver, a set top device, and a 
remote control; 



Figure 2 is an enlarged perspective view of the remote control of 
Figure 1; 



Figure 3 is a schematic block diagram view of certain elements of 
the set top device of Figure 1; 

Figure 4 is a schematic block diagram view of certain elements of 
the set top device of Figures 1 and 3 ; 

Figure 5 is a schematic block diagram view of certain elements of 
the set top device of Figures 1, 3 and 4; 

Figure 6 is a perspective view of another embodiment of the present 
invention which includes a television receiver and a remote control; 

Figure 7 is a perspective view of another embodiment of the present 
invention which includes a personal computer system and accessory 
input/output devices ; 



Figure 8 is an exploded perspective view of certain elements of the 
personal computer system of Figure 7; 

Figure 9 is a schematic block diagram view of certain elements of 
the personal computer system of Figure 8; 



Figure 10 is an illustration of the structure of a control program 
functioning with the systems of Figures 1 through 9 in accordance with 
this invention; 



Figure 11 is an illustration of the coding of a control program 
constructed using the structure shown in Figure 10; and 

Each of Figure 12 through Figure 18 is a view of the display screen 
of a television receiver of Figures 1 or 6 or personal computer system of 
Figure 7 operating in accordance with this invention. 



Detailed Description of the Invention 



Before undertaking a detailed description of specific embodiments 
of the present inventions, it is believed useful to set forth some 
description of the environments in which the inventions find utility. 



In more expansive forms, the inventions are practised using systems 
which have a video display device, circuitry for driving a display of 
visual images by the video display device, a display controller, and a 
remote control- In simplest form, the present inventions may be 
practised through the use of a remote control device and a display 
controller. 

Video display devices useful in the practice of the inventions here 
described are contemplated as including glass envelope cathode ray tubes 
(CRTs) such as are conventionally used in consumer electronics systems 
such as television receivers and in personal computer systems, television 
projectors such as are used in large audience displays, liquid crystal 
displays (LCDs) similarly used, gas plasma displays, and other flat panel 
displays. The listed types of devices are given as examples only, as it 
is contemplated that the types of displays with which these inventions 
are useful will extend to include still other types of display devices 
either not in common use or unknown at the time of writing this 
description, yet capable of displaying visual images to a human observer 
in a manner similar to the displays presented by the listed devices. 

in any instance, the display will be coupled to circuitry capable 
of delivering to the video display device video signals which drive the 
video display device to display such visual images. Such circuitry may 
include analog or digital tuners for receiving video signal streams 
transmitted or distributed at frequencies which are outside direct 
sensing by the human observer and which carry data which is to generate, 
after appropriate processing, the visual displays. Specific examples of 
such circuitry will be given hereinafter. However, it is contemplated 
that the circuitry may include that typically found in a set top device 
used as an accessory to a television receiver, in a television receiver, 
in a personal computer system, or in other types of consumer electronic 
systems . 

Video signal streams delivered to and through such circuitry may 
have a variety of characteristics. The streams may be of compressed 
signals, in which some information has been condensed or compressed by 
processing to facilitate transmission or storage. One set of such 
compression technologies are those specified by the Moving Pictures 
Experts Group (MPEG) . In such event, the circuitry may include provision 
for decompression of the video signal strecjn. The streams may be of 
uncompressed signals. The streams may be of analog information, such as 



conventional NTSC or PAL broadcast television quality, or of digital 
information derived from digitizing analog information or by direct 
authorship. The streams may be "live" in the sense of being transmitted 
and received and displayed concurrently with the occurrence of the events 
depicted, or recorded. Distribution of the signals may be by broadcast 
or by some broadband distribution method such as cable, optical fibre or 
the like. 

in all embodiments of these inventions to be here described, the 
video signal streams are delivered to the video display device under the 
control of a display controller. The display controller, as described 
more fully hereinafter, may be found in a number of different 
environments, now to be described. 

One such environment is provided by set top devices which, as 
contemplated by this invention, may be in the form of cable tuner 
systems, such as are used in many homes to which video streams are 
delivered by cable distribution networks. Set top devices may have the 
capability of decoding satellite transmissions, or video signal streams 
distributed in digital form, with or without encryption. They may also be 
in the form of devices which include record/playback capability, such as 
VHS tape or videodisc. They may also be in the form known as game 
machines, of which the systems offered by Nintendo and Sega are perhaps 
the best known. They may include back channel capability, so as to return 
a signal to a distribution system, either directly over a distribution 
link or through an alternate channel such as a conventional telephone 
line. A set top device may include some or all of the capabilities of 
the systems briefly mentioned above, as well as others perhaps not here 
set out in such detail. 

One such set top device is illustrated more specifically in Figure 
1, where are shown a television receiver 10, a remote control 20, and a 
set top device 30. The television receiver 10 is preferably a device of 
the type available to any consumer from any supplier of television 
receivers, and will have a housing or cabinet 11 within which is arranged 
a video display device 12. As described hereinabove, the display device 
12 may take any one of a number of forms. Also housed within the housing 
or cabinet 11 is video reception circuitry (not shown in Figure 1) which 
is coupled to the video display device for receiving signals transmitted 
at frequencies which are outside direct sensing by a human observer *.nd 
for delivering to the video display device video signals which drive the 




6 

video display device to display visual images perceivable by the human 
observer. The television receiver may be one configured to receive 
broadcast signals of NTSC or PAL standards or a "cable ready" receiver 
which implements a design capable of directly receiving a larger number 
5 of channels of analog signals such as may be distributed by a cable 

service provider. The television receiver may be one configured to 
receive a digital data stream, although at the time of writing of this 
disclosure such sets are not readily available commercially as a consumer 
product. Details of circuitry for such receivers may be found in any of 
10 a number of industry reference texts. 



The video reception circuitry is contemplated as being capable of 
receiving signals which carry analog information defining visual images 
to be displayed; digitally coded information defining such visual images; 
15 or compressed digitally coded information defining such visual images. 

Such signals as contemplated as being transmitted by broadcast 
transmission or by cable transmission or by satellite transmission or by 
transmission through a telecommunications network. 

20 One form of remote control is shown in Figures 1 and 2 at 20. 

Preferably, the control 20 is a three axis remote control device usable 
at some distance of separation from the television receiver 10. The 
meaning of the phrase "three axis" will become more clear from discussion 
which follows later in this description. The control 20 has a housing 21 

25 sized to be held in the hand of a human observer of the images displayed 

on the display device. The housing, while shown to be of a configuration 
particularly intended to lie comfortably in the hand of a user, may take 
any configuration which is reasonably held. The control 20 also has a 
manually engageable input device 22 mounted in the housing 21 for 

30 manipulation by the human observer and control transmitter circuitry (not 

visible in Figure 2) mounted in the housing and coupled to the input 
device 22 for transmitting at a frequency which is outside direct sensing 
by the human observer command signals coordinated in a predetermined 
manner to manipulation of the input device 22 by the human observer. 

3 5 Such circuitry, while not shown, may be as used in other more 

conventional hand held remote control devices such as are widely used by 
consumer electronic systems such as television receivers and audio 
systems. As such, the circuitry may follow the teachings of 
manufacturers of such devices. 

40 
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The "three axis" characteristic of the input device can also be 
known as a "press to select" characteristic. Stated differently (and as 
will become more clear as this description proceeds), the input device 
may be manipulated from side to side, toward and away from the user's 
hand, and toward any point around a circle centred on the device 22. if 
such actions were considered as if oriented to a compass rose, side to 
side motion might be toward and away from East and West, while motion 
toward and away from the user's hand might be toward and away from North 
and South. in this analysis, the device 22 is capable of indicating 
movement toward any point of the three hundred sixty degrees of the 
compass . 



when so manipulated, the input device 22 will generate signals 
which, in the contemplation of this invention, will ultimately give 
effect to movement of a cursor or pointer display element across the 
field of view provided by the display device 12. Once such manipulation 
has positioned the pointer over an appropriate portion of the visual 
images displayed (as will become more clear from discussion which 
follows), then an action indicated by such an element may be selected by 
pressing on the input device 22. Thus movement to points of the compass 
rose (as discussed above) is movement on two axes, while pressing on the 
input device 22 is movement along a third axis. It is the two axis 
movement for pointer positioning and third axis movement for action 
selection which gives rise to the terminology "three axis" remote control 
25 device. 

The input device 22. while shown in one form, may take a variety of 
forms. in particular, the device 22 is shown as what is here called a 
"wiggle stick", a wiggle stick, in the contemplation of this invention, 
is an elongate member pivoted within the housing 21 of the remote control 
20 and protruding therefrom. By suitable sensors, which may be strain 
gauge type devices or other electromechanical sensors, pressure exerted 
on the wiggle stick or physical movement thereof are transduced into 
electrical signals indicating manipulation by the human observer. 
Alternate forms of the input device 22 may be a wobble plate (similar to 
the device found on commercially available game controllers used with 
game machines accessories for television receivers), a trackball, a 
mouse, or an inertial mouse. The latter two forms of devices differ in 
that a mouse, as conventionally used with personal computer systems, 
rests upon a surface over which it is moved by a user to generate signals 
effecting movement a cursor or pointer display element across the 
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field of view provided by a display device while an inertial mouse 
references to a self contained inertial platform and may be manipulated 
free of a surface, as in the air. Such a device is also known as an air 
mouse . 

The remote control device 20 is coupled to the display controller 
(discussed in greater detail later in this description) in one of a 
variety of manners. In the form illustrated in Figures 1 through 6, the 
input device 20 is coupled by command transmitter circuitry mounted in 
the housing 21 and coupled to the input device 22 for transmitting at a 
frequency which is outside direct sensing by the human observer command 
signals coordinated in a predetermined manner to manipulation of the 
input device by the human observer. Such command signals, as is known to 
persons of skill in the arts related to other pointer control devices, 
may be emitted by an infrared radiation emitter, a radio frequency 
emitter, or an ultrasonic emitter. In other forms, described hereinafter 
in connection with the personal computer system of Figures 7 through 9, 
command signals may be transferred through an elongate flexible 
conductor . 

One form of set top device 30 is more particularly shown in Figures 
3 through 5 and will be described in some detail with reference to those 
Figures. However, it is to be recognized that the particular device here 
described is only one of a number of varieties of such devices as alluded 
to hereinabove. The illustrated embodiment preferably has an analog 
multiplexer 31 through which many of the signals flow among elements of 
the device 30 as illustrated in Figure 3. Signals reaching the analog 
multiplexer 31 can arrive from an antenna or cable connection 32 through 
first or second tuners 34, 3 5 or a cable interface 36. The cable 
interface may allow for decryption of securely encoded signal streams, 
either on a single use ("Pay per view") or timed interval (subscription) 
basis. The analog multiplexer 31 also serves as a conduit for signal 
streams from the output of an MPEG processor 38, the video processor 39, 
a video recording/playback device 40 such as a VHS video cassette 
recorder/player or a videodisc player, and auxiliary devices such as a 
camera (not shown) through a camera auxiliary port 42 or a game machine 
(not shown) through a game auxiliary port 44. 

The video processor 39 is a central element of the set top device. 
In addition to the elements recited above, the processor 39 is 
operatively connected with system memory 45, an analog audio control 46, 



a microprocessor 48 functioning as a central processing unit or CPU, 
flash ROM memory 49, an I/O processor 50 including an infrared 
receiver/blaster, an expansion bus 51, a cable or telephone modem 52, anc 
a Compact Disk (or CD) drive 54. Each of these elements serves function: 
to be described more fully hereinafter. 

The video processor 39 will be discussed in detail in the text 
which addresses Figure 5. Suffice it to say for now that the video 
processor 39 comprises the following functional blocks: dual VBI 
decoders, sync extract, genlock and timing, a memory refresher, a video 
controller, a blitter graphical coprocessor, a CD drive controller, a 
digital signal processor (DSP) sound coprocessor, and an arbitrator to 
arbitrate the access to the system memory between the six possible bus 
masters (the CPU, the blitter, the DSP, the memory refresher, the video 
controller, and the CD drive controller) . Referring to Figure 3, the 
arbitrator controls the changing priorities of the devices, as described 
herein, and is in electrical circuit communication with all the devices 
within the video processor 39. For example, the CPU 48 has the lowest 
priority of all bus masters until an interrupt occurs. Thus, the 
arbitrator is in circuit communication with both an interface to the CPU 
and an interrupt controller. 



The two tuners 34 and 35 of system 30 can be configured to support 
standard broadcast or cable based input (received from the antenna or 
cable connection 32) and produce standard analog composite video output. 
Both tuners 34 and 35 receive the full RF broadcast spectrum from antenna 
32 but each tuner typically tunes to different frequencies. (The term 
"composite" video used herein refers to an amplitude modulated signal 
which contains all the information necessary to display a moving picture 
all on a single wire. This includes the horizontal and vertical 
synchronization pulses (timing) , colour synchronization pulses 
(chrominance) and individual pixel intensity (luminance) and colour for 
each line of the display.) Channel selection and other tuner parameters 
are controlled via a serial interface which could be well known serial 
protocol for consumer products tic (l 2 C) . The tuners 34 and 3 5 can be 
those that are available from Philips (P/N F11236) and other suppliers. 
Phillips also supplies the TEA5582 stereo decoder which takes the single 
tuner output and converts it into separate left and right audio signals. 



The tuner outputs are connected to an optional cable interface 
module 36 which can be supplied by the cable operator. Each cable 
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company may have a unique system for scrambling their premium channels 
which will require a unique set of hardware to reconstruct the signal 
into standard composite video for subsequent processing and display. The 
cable interface 36 can support descrambling of two video signals so that 
the user may watch and/or record two premium channels simultaneously. 

A video recording/playback device 40 (e.g., VCR or video disc 
player) which cam be built in to system 30 provides an input source of 
composite video and audio. The Play, Record, Rewind, Pause, Fast 
Forward, Frame advance and other features of the VCR 4 0 are controlled 
via a serial interface which could be IIC or any other low cost protocol . 
Serial interface controlled vCR's are available from Goldstar, JVC, 
Panasonic and other suppliers. 

An optional MPEG decompression module 3 8 can be added to the system 
30 which will also produce a composite video and audio signal. MPEG 
decompression chips are available from IBM, Texas Instruments (TI) , SGS, 
Thompson, C-Cube, and other suppliers. 

Composite video and audio RCA input jacks conveniently support 
playback of camcorder, camera or other input sources which can be 
supplied to system 3 0 on the AUX1 connection 42. In addition, composite 
video and audio RCA input jacks also support game players, laser disk 
players or any other sources which can be supplied to system 3 0 over the 
AUX 2 connection 44. 

The analog video multiplexer 31 is the central switching point for 
composite video and stereo audio processing in the system 30. Each of 
the three outputs of the multiplexer 31 are completely independent and 
can select from any of the seven inputs shown in Fig. 3. The selection 
is made via the control microprocessor 4 8 under control of the operating 
system and user interface. As will be described in more detail below, 
the selection can be made by a user via the remote control 20. The user 
would be presented with a graphical representation of his video options 
and he would point and select to determine which of the seven sources go 
to which of the three destinations (full screen, PIP, VCR) . For example, 
when a user selects to view channel 5 in a PIP over channel 8, the user 
interface code will program the full screen output of multiplexer 31 to 
come from tuner 34 (which is set to channel 8) and the PIP output of 
multiplexer 31 will be set up to come from tuner 35 (which is set to 
channel 5 ) . 
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One output of the multiplexer 31 is sent to the video processor 39 
and will be processed as the full screen picture. This signal is kept i 
an analog composite video format in the video processor 39. This signal 
horizontal sync, vertical sync and chroma subcarrier are used as. a 
reference to genlock the graphics and Picture in Picture { PIP) overlay i 
the video processor 39. A second output of the multiplexer is also sent 
to the video processor 3 9 but as the video signal to be converted to 
digital and scaled to a small (e.g., 1/9 to 1/4 full screen) live video 
window of an alternate channel to be overlaid onto the full screen video 
signal. A third output of multiplexer 31 drives the VCR 40 such that th« 
VCR 40 is able to record any one of the seven inputs. It should be note, 
that, the composite video output of the processor 39 is also available a.' 
an input to the internal VCR 40 (via multiplexer 31) for recording syster 
generated graphics overlays and Pip. 

The CPU 48 has a SYSTEM bus associated with it. The SYSTEM bus 
includes a DATA bus, ADDRESS bus, and CONTROL bus. The video processor 
39 is the arbitrator for the system memory 45; therefore, the SYSTEM bus 
is modified to a SYSTEM' bus (comprising a DATA' bus, ADDRESS ' bus, and 
CONTROL' bus) by the video processor 39. 

The system memory 45 comprises screen RAM, system RAM, and 
bootstrap ROM. The system memory 4 5 is an expandable single memory bank 
that supports the system processor 48 and video memory requirements. The 
single memory 45 supports all data used by system 30 including system 
data, video data, graphics data and electronic program guide data. The 
single memory model allows dynamic balancing of memory between the system 
processor 48 and the video processor 39. The system memory 45 will be 
discussed in more detail in the text accompanying Figure 5. 

The flash memory 49 is used to store the operating system, user 
interface code, application programs and user data. Flash memory 49 
includes at least 512K bytes of flash expandable to 1M byte. Flash 
memory 49 is preferred over PROM or EPROM to allow new operating system, 
user interface and applications to be downloaded from a central server. 

The I/O processor 50 interfaces the CPU 48 to numerous I/O devices, 
such as the remote control 20, a keyboard, a digitizer, a printer, or a 
touchpad. in a preferred embodiment, the I/O processor is a 
preprogrammed MC68HC705C8 (hereinafter » 6 8HC705 <• ) , manufactured by 
Motorola Corp, running at 2 MHz . The 68HC705 I/O processor is interfaced 
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to the CPU 48 by configuring the 68HC705 as a peripheral device: 
(1) PA0-PA7 are connected to D0-D7 of the DATA bus; (2) PB7 , PB1, and PB2 
are connected GPI01 (a 32 -byte address range decoded by the video 
processor 39), Al, and A2, respectively, of the ADDRESS bus and CONTROL 
bus; and (3) PB3 , PB4 , and PBS are connected to ADS, READY, and W/R, 
respectively, of the CONTROL bus. Thus, the I/O processor is decoded to 
have four 16 -bit addresses in I/O space (referred to herein as ASO, AS2 , 
AS4, and AS 6) . The I/O processor also interfaces with appropriate 
receiver circuitry which is able to detect and receive the signal packets 
emitted from the remote control 20. The I/O processor 50 also sends 
commands to other infra red devices to allow the single user interface to 
drive other video and audio equipment for the user. 

The program inside the 68HC705 interfaces to the CPU 48 as follows. 
The 68HC705 is designed to attach directly to the processor bus and act 
as an I/O port to the CPU 48. A pair of internal latches hold data 
passing between each of the processors until the other is ready to 
receive it. Status bits to each processor indicate the condition of the 
data latches. Each can tell if the previous data has been read and if 
any new data is waiting to be read by checking the status bits. 

The I/O processor 50 implements the following functions: (1) a 50 
ms timer, (2) a serial controller link for input devices, (3) a system 
reset, and (4) a data/strobe/acknowledge { DS A) CD control communications 
link for the CD drive 54. 

The 5 0 ms timer is implemented using the watchdog timer of the 
68HC705 I/O processor. when the watchdog timer expires, the I/O 
processor 50 interrupts the CPU 48 using analog interrupt 1 (All) of the 
video processor 39. The CPU 48 responds to this by reading the 16 -bit 
I/O port ASO, described above, which causes the video processor 39 to 
activate the I/O processor 50, thereby causing a data transfer between 
the CPU 48 and the I/O processor 50. 

Input devices are connected to the I/O processor 50 via a serial 
controller link and controllers. The controllers transform the signalled 
movements of control devices into a format suitable for transmission 
along the serial link. The controllers send data packets via the 
controller serial data link to the system unit. The data packets differ 
depending on the type of 10 device. Co-ordinate type devices (such as 
those with which the present invention is particularly concerned 
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including a wiggle stick, wobble plate, mouse, joystick, etc.) have a 
different data packet than a switch closure type of device (keyboard, 
digital joystick, switch pad. etc) . The controllers will include 
receivers appropriate to any signals emitted by a remote control device 
5 20. such as infrared receivers, radio receivers, etc. 

The serial controller link consists of three (3) lines: a data 
receive line, a VCC (+5 VDC) line, and a ground line. The 68HC705 
implements the data receive line of the controller serial link using the 
10 PDO/RDI pin. This pin is designed to be used as an interface to serial 

devices using the well known asynchronous format. A clocked synchronous 
format could be used in the alternative. 
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An expansion bus 51 can be a buffered 16 bit microprocessor local 
bus for entry level products or a 32 bit peripheral component 
interconnect (PCI) bus for higher performance systems. The expansion bus 
51 reduces the cost of the system 30 while allowing a growth path for 
additional features. 



System 30 also includes a modem 52 for providing bi-directional 
communications necessary for information services and software updates. 
The modem 52 can be a phone modem using the digital signal processor 
built into the video processor 39 and additional analog logic. 
Alternatively, the modem 52 can be a cable modem (such as 64 QAM or 16 
25 V SB) for bi-directional cable support. 

The system 30 can include an optional CD-ROM drive 54 that may be 
used to play audio, digital MPEG movies and data CDs. The CD drive 54 
provides a serial data stream to the video processor 3 9 where it is 
3 0 decoded and processed. The CD drive 54 is fully concurrent with the 

video processing. The control for the CD drive 54 is provided via the 
IIC interface or other serial protocol. 



As alluded to hereinabove, the CPU 48 generates multiple buses: a 
DATA bus, ADDRESS bus, and CONTROL bus, as are well known in the art. 
These three buses are collectively referred to as the SYSTEM bus. in the 
preferred embodiment, the CPU 48 is an 80376, manufactured by Intel 
Corp., 3065 Bowers Ave., Santa Clara, California, 95051. The 80376 is a 
variation of the well known 80386SX, which is well known in the art and 
also available from Intel Corp. The 80376 differs from the 80386SX in 
that the 80376 starts up in 32 -bit mode, rather than 16 -bit mode. 
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Specif ically / the CRO register is forced to a 0011H (0011 in hexadecimal 
notation) state with bit 0 forced to a logical ONE, effectively making 
the 376 operate in a 32 -bit memory mode. Paging is enabled to allow 
virtual 386 operation. 

5 

The present invention contemplates that the CPU 4 8 may access 
control programs stored, for example, in the set top device system memory 
45 so as to be accessible to the processor, for controlling the display 
of visual images by said video display device 12. As will be understood 

10 by persons of skill in the design of program controlled digital devices, 

the processor accessing such a control program will be capable of loading 
the control program and operating under the control of the control 
program so as to accomplish the functions established by the author of 
the program. Such a control program may, for example in this disclosure, 

15 cause the command receiver circuitry associated with or embedded in the 

I/O processor 50 which receives command signals from the command 
transmitter circuitry of the remote control 20 to derive from the 
received command signals image directing signals directing modification 
of visual images displayed on the display device. Further, the control 

20 program will cause command processor circuitry in the video processor 39 

which is coupled to the command receiver circuitry and to the video 
reception circuitry in the television receiver 10 to receive the image 
directing signals and modify the visual images displayed on the device 12 
as directed by manipulation of the remote control by a human observer. 

25 

In executing control programs, the systems here described will 
receive and store and deliver digitally encoded data in memory devices 
and execute in a microprocessor coupled to the memory devices digitally 
encoded control programs stored in the memory devices. The control 

30 programs will be effective on execution by the microprocessor for 

modifying video signals in predetermined manners in response to 
predetermined image directing signals derived from manipulation of the 
remote control 20. Such execution of a control program will include 
controlling microprocessor access to operational resources of the 

35 television video display device by execution of an operating system 

program and/or controlling modification of the video signals by execution 
of an application program. That is, the control exercised is based upon 
both operating system allocation of resource access and application 
program utilization of accessed resources. 

40 
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Additional circuitry 56 associated with the set top device 3 0 is 
shown in Figure 4. Referring now to Figure 4, the additional circuitry 
comprises two devices: an RF modulator 58, and an audio analog- to-digita 
converter/ digital • to -analog converter/compressor/decompressor 
(ADC/DAC/CODEC) 59. 

The video processor 39 has a number of functional blocks that will 
be more fully described in the text accompanying Figure 5. It is 
sufficient for this point in the description to note that two such block, 
are a video controller 60 and a digital signal processor (DSP) 61. 

An RF modulator 58 merges the composite video signal from the videc 
processor 39 with the left and right audio line out signals from an audit 
ADC/DAC/CODEC 59 onto a carrier frequency to generate an rf video signal, 
indicated by RF video, suitable for being directly input into the 
television receiver 10. To generate the different pal (European 
television standard) and NTSC (US/Japanese) television signal standard 
formats, a different RF modulator and crystal may be used. The RF video 
signal is connected to external devices with a single female Type F 
coaxial connector, as is well known in the art. 

The audio ADC/DAC/CODEC 59 is linked to the DSP 61 with a serial 
link conforming to the well known Philips I 2 S protocol. The 
ADC/DAC/CODEC 59 converts analog data to digital data, and vice versa, 
and compresses and decompresses digital data. The ADC/DAC/CODEC 59 
interfaces external stereo analog data from optional microphones to the 
video processor 39. The audio inputs are connected to external devices 
with a standard stereo 1/4- connector. The audio ADC/DAC/CODEC 59 also 
interfaces digital data from the video processor to external devices by 
generating left and right audio line out signals. These signals are 
connected to external devices, such as optional speakers with two female 
RCA phone jacks, as are well known in the art. As mentioned above, the 
audio line signals are also added to the RF video signal. 

In the preferred embodiment, the ADC/DAC/CODEC 59 is a CS4216, 
manufactured by Crystal Semiconductor. The part contains microphone 
inputs, with programmable gain, as well as outputs with programmable 
attenuators. Gain and attenuation are both programmably controlled by 
the DSP 61. 



40 
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In the alternative, the ADC/DAC/CODEC 59 can be replaced with a 
TDA1311 DAC manufactured by Philips. If this chip is used, the ADC and 
CODEC functions will not be available. 

5 Referring now to Figures 3 through 5, the video processor 39 

electronics are largely contained within one massive custom logic chip, 
known as an ASIC (Application Specific Integrated Circuit) . As 
illustrated in Figure 5, the video processor contains a processor 
interface 62, a processor cache 64, a memory interface/refresh 66, a 

10 video controller 60, an interrupt controller 70, a video blitter 72, a CD 

drive controller 74, a digital signal processor (DSP) 61, a DSP memory 
76, two VBI decoders 80, 81, a multi - standard decoder 82, a scaler 83, a 
sync extract circuit 84, genlock and timing 85, a multi - standard encoder 
86 and a composite video multiplexer 88. The processor interface 62, the 

15 memory interface/refresh 66 (DRAM controller) , and the video controller 

60 are referred to collectively as the video/memory controller 78. The 
system memory 45, central processing unit 48, and other devices lie 
outside the video processor 39. 

20 The SYSTEM' bus electrically connects the various devices to the 

system memory 45. Sharing the SYSTEM' bus are six possible bus masters 
(in order from highest priority to lowest priority, respectively) : the 
memory refresh 66, the video controller 60, the CD drive controller 54, 
the DSP 61, the blitter 72, and the CPU 48 (through the processor 

25 interface 62) . Only one of the bus masters may control the SYSTEM' bus 

(DATA' bus, ADDRESS ' bus, and CONTROL ' bus between the video processor 39 
and the system memory 45) at any one time. 

The video/memory controller 78 controls the SYSTEM' bus, and 
3 0 provides the memory timing signals (for example, CAS, RAS, write enable, 

etc.) for memory devices attached to the SYSTEM' bus, as is well known in 
the art. It also requires memory cycles (video memory cycles are 
required to read video data from system RAM; since video is generated in 
real time by this process, the video logic must have memory access when 
35 video data is needed) , and has effectively the highest priority on the 

SYSTEM' bus, as mentioned above. It suspends bus master operations 
during video lines for brief periods to fetch any video display data, and 
to refresh dynamic RAM (DRAM) . It also controls the interface with the 
CPU 48. 

40 



BN80OOD: <QB_2310336A-JU> 



17 



The video processor 39 receives up to the two analog composite 
video signals (FULL SCREEN and PIP) from the multiplexer 31. The FULL 
SCREEN image is maintained as analog composite video and processed as the 
full screen image while the PIP analog composite video signal is input t< 
the video processor 39 as a full screen image and overlaid onto the full 
screen by processor 39 as a picture in picture image. The video 
processor 39 also overlays system generated graphics onto the video and 
outputs the composite to the display device 12. 

The video processor 39 can also receive a CD data stream from the 
CD drive 54 which is decoded and placed into DRAM 45. The CD data may be 
MPEG encoded movies, photo CD, data CDs, audio or program data. The DSP 
61 can implement a 1200 baud send or a 2400 baud receive FAX modem. A 
high speed blitter 72 supports rapid transfer of video sprites from one 
section of DRAM 45 to another which enhances user interface and 
application appearance and aids in data movement from the CD controller 
54. A cache controller 64 improves system performance for the host 
microprocessor 48 since DRAM bus bandwidth is heavily loaded with 
graphics and live video access. 



The sync extract circuit 84 receives the FULL SCREEN composite 
video input signal from multiplexer 31, extracts the horizontal (HSYNC) 
and vertical (VSYNC) synchronization signals as well as the chroma 
subcarrier from the FULL SCREEN composite video input and provides the 
HSYNC signals, VSYNC signals and chroma subcarrier to the genlock circuit 
85 via line 89. In addition, the sync extract circuit 84 also passes the 
original FULL SCREEN composite video input to multiplexer 88 via line 87. 
If no signal is present on the FULL SCREEN composite video input to the 
video processor 39, a sync and fixed colour (e.g., blue) screen is 
generated automatically by a crystal based local oscillator and timing 
circuit within genlock circuit 85. 

The genlock and timing circuit 85 will utilize the horizontal and 
vertical syncs (as will be described below) and passes the chroma 
subcarrier to the encoder 86 to ensure that the colour reference for the 
PIP is the same as the full screen image. The genlock and timing 
generator 85 builds a dot clock for the video controller 60 which is 
phase synchronous to the full screen video horizontal sync. This is 
accomplished by feeding a 15.734 Khz horizontal sync that has been 
extracted from the full screen video signal (by sync extract circuit 84) 
into the reference input of a phase locked loop. A comparator input to 
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the phase locked loop is the output of a voltage controlled oscillator 
(VCO) {within genlock 85) divided by 3185. The output of the VCO also 
forms the dot clock for the video controller 60 which is 14 times the 
frequency of the chroma subcarrier ( 3 . 57954 545Mhz) . The vertical sync 
for the video controller 60 is established under software control by 
resetting the video controller's vertical line counter. The horizontal 
sync frequency, chroma subcarrier frequency and division given are for 
NTSC, corresponding values being required for PAL operation- 



First and second vertical blanking interval (VBI) decoders 80 and 
81 extract electronic program guide data, channel violence ratings, time 
remaining, close captioning for the hearing impaired and other digital 
data from the vertical blanking interval on fields one and two of lines 
ten to twenty-one of the NTSC protocol. This data can be extracted from 
15 either the FULL SCREEN composite video input by the first VBI decoder 80 

or from the PIP composite video input by the second VBI decoder 81 
individually or simultaneously by control of the microprocessor 48. The 
digital data is selected, decoded and stored under operating system 
control in the system, video and graphics shared dram 45. Besides the 
20 standard use for close caption display the system could be configured to 

use one or both of the VBI decoders 80, 81 to scan the closed caption 
text of a channel for key words and alert the user or record the program 
when items of interest are discovered. 

25 A multi -standard (NTSC/PAL) ("PAL" referring to the well known 

European television signal standard) video decoder 82 receives the PIP 
composite video input where it is first digitized via a 25-30Mhz sampling 
A/D decoder in video decoder 82 and then converted to YUV colour space. 
The decoder 82 extracts the colour phase difference signal for each pixel 

30 and converts them into YTJV values with Quarter CIF (176 x 144) 

resolution. Limited resolution reduces the memory required for the frame 
store yet still provides minimum function required by the user. 

A scaler 83 receives the digital video signal output by decoder 82 
35 and reduces the size of the PIP video signal to between 1/4 and 1/9 full 

screen and transfers the reduced digital video signal to the system, 
video and graphics shared DRAM 45 via the video/memory controller 78. 

The DRAM controller 66 (described in detail below) provides the 
40 decode, interface and refresh signals necessary to support standard DRAM. 

The DRAM controller 66 also arbitrates requests for DRAM access among the 
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video processor 39 functions. The video controller 60 is given highest 
priority with second priority going to the scaler 83. 

The video controller 60 rasterizes the graphics and pip image from 
DRAM 45 for output to the multi- standard composite video encoder 86. 
More specifically, the video controller 60 takes the digital 
representations of the image in DRAM 45 and paints the image one pixel at 
a time to the encoder 86. The video controller 60 paints the pixels to 
the encoder 86 at just the right time so that the PIP image will overlay 
the full screen image at the appropriate spot. The video controller 60 
supports resolutions as high as 640 x 480 with 16 bits per pixel 
interlaced or non-interlaced. The DRAM controller 66 and video 
controller 60 will be more fully described in the text below with regard 
to video/memory controller 78. 



The multi -standard encoder 86 receives red, green, blue (RGB) 
information from the video controller 60 and synchronization information 
(HSYNC, vsync and chroma subcarrier) from the genlock and timing circuit 
8 5 and converts the data into PIP composite video data for output to the 
multiplexer 88. The multi -standard encoder 86 includes a video digital 
to analog converter (video DAC) and an NTSC/PAL encoder. The video 
controller 60 of the video processor 39 connects to the video DAC , which 
converts eighteen bits of pixel information (six bits each of red, green, 
and blue) from the video controller 60 into an RGB signal, as is well 
known in the art. Each colour channel (R, G, and B) of the video DAC is 
implemented with an R2R resistor tree and a 2N2222 transistor. The RGB 
signal is converted to composite video with the NTSC/PAL encoder. The 
encoder receives chroma subcarrier, HSYNC and VSYNC from genlock 85 and 
generates an NTSC composite video signal from the RGB outputs of the 
video controller 60. In the alternative, the well known PAL (European 
television signal standard) format can be generated. The composite video 
signal from the encoder 86 is supplied to multiplexer 88. To ensure 
consistent colour of PIP and graphics images, the chroma subcarrier for 
the encoder 86 is derived from the full screen image as described above. 
35 In the Preferred embodiment, the NTSC/PAL encoder 56 is an MC1377, 

manufactured by Motorola Corp. 

The DSP 61 is a simple, very high-speed processor for sound 
synthesis, operating at up to 33 million instructions per second (mips) . 
40 Tt has access to the SYSTEM' bus via a DSP DMA controller (not shown), 

which allows it to read and write bytes or words into system memory 45. 
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These transfers occur in short bursts, and are under DSP program control. 
The DSP 61 actually executes programs and stores data in its own private 
high-speed memory 76. 

The compact disk read DMA channel of the CD controller 54 allows 
the system to transfer CD read data into system memory 4 5 without any 
software overhead. It may transfer data directly; it also contains a CD 
block decoder. 

The interrupt controller 70 interfaces six internal interrupts to 
the CPU 48: video interrupt (highest priority), analog interrupt 1 
(All), analog interrupt 2 (AI2) , analog interrupt 3 (AI3), CD block 
decoder interrupt, and DSP interrupt (lowest priority) . The interrupt 
controller automatically clears an interrupt when the CPU 48 performs the 
interrupt acknowledge cycle. A mask bit is available for each of the 
interrupts . 

The blitter 72 is a graphics processor for fast screen updates and 
animation, acting as a hardware graphics subroutine for the CPU 48 or 
DSP 61. It will become bus master through blitter program operation, and 
may therefore own the SYSTEM' bus for considerable periods. However, its 
priority over the CPU 48 is not absolute; it may be requested to give up 
the SYSTEM' bus to the CPU 48 when an interrupt occurs. The CPU 48 is 
the lowest priority bus master at the system level; however, it has 
complete control of the other hardware, therefore, the use of the SYSTEM' 
bus is entirely under CPU 48 program control. 

The video processor 39 has several major blocks: a video/memory 
controller 78, sync extract 84, genlock and timing 85, a compact disc 
controller 54, a blitter graphics coprocessor 72, and a DSP audio 
coprocessor 61. The address space of the CPU 48 is decoded to a number 
of eight -bit registers within the video processor 39. All internal 
locations are on even address boundaries; word -wide I/O reads and writes 
may be performed where appropriate. In this particular embodiment, the 
byte -wide writes may not be performed on word- wide registers and I/O 
cycles may not be used to access odd addresses. 

In addition to the above registers, the video processor 39 
generates three spare general purpose I/O decoder lines (GPIOI, GPI02, 
and GPI03) from the SYSTEM bus, each providing a 32 -bit I/O address 
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range. The general purpose decoders may be used to provide three active 
low chip enables to devices external to the video processor 39. 

The video/memory controller 78 performs several functions: video 
timing, interrupt handling, video display generation, and memory 
configuration, refresh, and timing (the latter three functions are 
performed by DRAM controller 66) . 

The video/memory controller 78 has a flexible video timing 
generator that can be programmed to suit different TV standards and 
monitors up to a 64 0 by 4 80 VGA standard. The position of 
synchronization pulses, blanking, display area, active video (when the 
video processor 3 9 is fetching data from memory) are programmed in clock 
cycles in the horizontal dimension and in line numbers in the vertical 
direction. Video timing is broken into two parts. Horizontal timing is 
defined in terms of clock cycles and is determined by a number of eleven - 
bit registers. vertical timing is defined in terms of display lines and 
is determined by a number of ten-bit registers. 

There are nine horizontal registers.- horizontal period, horizontal 
sync, horizontal blanking end, horizontal blanking begin, horizontal 
display begin, horizontal display end. horizontal fetch begin, horizontal 
fetch end, and horizontal vertical sync. The value written to the 
horizontal period register determines the horizontal line length in clock 
cycles. in one embodiment the line length is one greater than the number 
written to the horizontal period register. The formula for the required 
number is: horizontal period = (line length x clock frequency) - one. 

The value written to the horizontal sync register determines the 
width of the horizontal sync pulse. The width of horizontal sync in 
clock cycles is given by the difference between the horizontal period 
register and the horizontal sync register. The formula for the required 
number is: horizontal sync = horizontal period - (horizontal sync width 
x clock frequency) . The horizontal blanking end register determines when 
the horizontal blanking ends and is the width of the back porch in clock 
cycles. The horizontal blanking begin register determines where 
horizontal blanking begins. The formula for the required number is: 
horizontal blanking begin = horizontal period - ( (horizontal sync width * 
front porch width) x clock frequency) . 



40 



BNSOOC1D: <QB 2310336A_JU» 



The horizontal display begin register specifies how soon video is 
generated after the trailing edge of horizontal sync in clock cycles. If 
the horizontal display begin register is greater than the horizontal 
blanking end register the video /memory controller 78 outputs the border 
colour in-between. The value written to this register should normally be 
chosen to put the picture in the middle of the television screen. The 
formula for a register number to do this is: horizontal display begin = 
(horizontal blanking end + horizontal blanking begin - (active display 
width x clock frequency) ) /2 . 

The horizontal display end register specifies where the display 
ends and therefore determines the width of the video display in pixels. 
It should be programmed with the following number: horizontal display 
end = horizontal display begin + (number of pixels x clocks per pixel) . 
If horizontal blanking begin is greater than horizontal display end, then 
the border colour will be output until blanking begins. 

The horizontal fetch begin register determines where video fetches 
first start on the line. This should be programmed such that the sixteen 
byte pixel buffer has just been filled when the display begins. In 
practice, this means that the value in the horizontal fetch begin 
register is given by the value in horizontal display begin less a 
constant which depends on the display mode. The table below contains the 
constants for various combinations of bits per pixel and clocks per 
pixel. For example, if four bits per pixel and five clocks per pixel 
then the constant is 160. Likewise, if four bits per pixel and one clock 
per pixel, then the constant is 32. Note that if there are 16 bits per 
pixel and one clock per pixel, then no constant is applicable. 

Clocks per pixel 

f ive four three two one 

Bits per pixel four 160 128 96 64 32 

eight 80 64 48 32 16 

sixteen 40 32 24 16 n/a 

The horizontal fetch end register determines where video fetches 
end on the line. In principle, this is the value in horizontal display 
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end minus the above constant. However, horizontal fetch begin should be 
rounded up so that horizontal fetch end register minus the horizontal 
fetch begin register is a multiple of the above constant. 

The horizontal vertical sync is identified as wider sync pulses 
occurring on a number of lines. The width of these pulses is determined 
by the horizontal vertical sync register which should be programmed as 
follows: horizontal vertical sync = horizontal period - (vertical sync 
width x clock frequency) . 

The video/memory controller 78 also has a large number of vertical 
registers: the vertical period register, the vertical sync register, the 
vertical blanking end register, the vertical blanking begin register, the 
vertical display begin register, the vertical display end register, the 
video interrupt register and the light pen registers. The vertical 
period register specifies the number of video lines per field. The 
vertical sync register determines the number of lines on which vertical 
sync is generated. It should be programmed as follows.- vertical sync = 
vertical period - lines of vertical sync. 



The vertical blanking end register determines how many lines are 
blanked after a vertical sync. The vertical blanking begin register 
determines how many lines are blanked before vertical sync. It should be 
programmed as follows: vertical blanking begin = vertical sync - lines 
25 of blanking prior to vertical sync. 



30 



The vertical display begin register determines the first line of 
active video. If this register is greater than the vertical blanking end 
register the lines in-between show the border colour. To position the 
active area in the middle of the screen this register should be 
programmed as follows: vertical display begin = (vertical blanking end + 
vertical blanking begin - number of active lines) /2. The vertical 
display end register determines the last line of active video. if this 
register is less than the vertical blanking begin register the lines in- 
35 between will show the border colour. to position the active area in the 

middle of the screen this register should be programmed as follows: 
vertical display end = (vertical blanking end + vertical blanking begin + 
number of active lines) /2. 



40 



The video interrupt register determines the video line on which a 
video interrupt is generated. This interrupt may be enabled or disabled 
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through the INT register. The interrupt occurs when the video mechanism 
stops at the end of the display line. It may be used by the processor to 
change display modes or to perform beam synchronous animation. The 
register may be reprogrammed within a field to provide several interrupts 
5 per field. 



The following table provides typical values for the above registers 
for the various display formats shown. After loading the registers with 
the below values, the video timing generator is enabled by setting the 
10 viDEN bit in the register MODE2 . 
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5 0 Hz PAL 


60 Hz NTSC 


VGA 






320 x 256, 
8-bits 


320 x 220, 
8 -bits 


640 x 480, 
8-bits 




Clock frequency 


22 . 17 MHz 


21.48 MHz 


25.17 MHz 




Horizontal period 


1418 


1363 


790 


5 


Hori 2ont- a 1 cvnr 


1 j 14 


1262 


703 




Horizontal 
b ldnkina ^nri 


126 


103 


.48 




Horizontal 
blanking begin 


1271 


1232 


688 


10 


Horizontal 
disDlav Kpni n 


378 


348 


48 




Horizontal 


1018 


988 


688 


15 


Horizontal fetch 
beg in 


346 


316 


32 




end 


y 0 0 


9 56 


—— __ ____________ 

672 




Horizontal 
vertical sync 


103 


89 


0 


20 


Vertical period 


312 


262 


525 




v CI LlUdi £> y II c 


3 09 


259 


524 




vertical blanking 
end 


20 


15 


34 


25 


Vertical blanking 
begin 


307 


257 


514 




vertical display 
begin 


35 


26 


34 


30 


vertical display 
end 


291 


246 


514 
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The video/memory controller 7 8 has three colour resolutions 
available: four bits per pixel, eight bits per pixel, and 16 bits per 
pixel. In four- and eight -bit modes, the pixel is a logical colour that 
indexes an 18-bit physical colour stored in the palette. In 16 -bit mode, 
5 the pixel is a physical colour in which bits zero to four are blue, bits 

five to ten are green and bits 11 to 15 are red. Because there are six 
bits of green but only five bits of blue and red, the least significant 
bits of blue and red output from the chip are always logical ZERO in 16- 
bit mode. The border colour is a 16 -bit register which is displayed as a 
10 16-bit pixel. 

in eight-bit mode, the pixel addresses the whole 256 by 18 palette. 
In four -bit mode the pixel addresses 16 entries from the palette in which 
case the top four bits of the addresses are supplied from the index 
15 register. 

Two variations are available in eight -bit mode. In colour hold 
mode if the pixel takes the value zero, then the colour of the previous 
pixel is displayed. This can be used to fill large areas of colour 
20 simply by setting the left most pixel. 

In variable resolution mode, the most significant pixel determines 
whether the pixel is displayed as one seven -bit pixel or two three -bit 
pixels. If the bit is clear, the pixel is displayed as one seven-bit 

25 pixel; if the bit is set then bits zero to two are displayed first 

followed by bits four to six. In this case, the two high resolution 
pixels address eight entries from the palette. The top five bits of the 
address are supplied from the index register. Variable resolution mode 
is useful for displaying small regions of high resolution text amid a 

3 0 lower resolution, but more colourful, background. This mode is not 

available in one clock per pixel resolution. 

In eight -bit mode any of the bits can be sacrificed and used for 
other purposes. For instance, a bit could be used to identify "hot 
35 spots" for collision detection. Alternatively, bits could be used to 

encode image "depth" so that one image can move in front of or behind 
another. To sacrifice a bit, the same bit in a masked register is set 
and that bit will be replaced from the corresponding bit in the index 
register . 

40 
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There are five widths of pixel: one clock, two clocks, three 
clocks, four clocks, and five clocks. These correspond to dot clocks of 
around 24 MHz, 12 MHz , and 6 MHz . The highest dot clock may not be used 
with the 16 bits per pixel display mode. Two other combinations :., one 
clock 8 -bit and two clock 16 -bit may only be used if 32 -bit DRAM is 
fitted. If external hardware is fitted as will be in the applications 
here described, the video processor 39 can gen- lock to an external video 
source and mix (encrust) local video with external video on a pixel by 
pixel basis. This is significant with regard to certain displays to be 
generated in accordance with this invention as described more fully 
hereinafter. 

The memory map of the screen is not tied to the video display widtl 
but is defined independently. The base address of the screen can be 
anywhere in system memory 45. The width of the screen memory is the 
power of 2 from 128 to 2048 bytes. The height of the screen is a power 
of 2 from 32 K to 2 megabytes. Video addresses on the same line wrap 
within the smaller boundary. This arrangement allows the screen to be 
placed within a larger virtual screen and panned and scrolled within it. 

various registers control the video modes discussed above. 

The video mode register controls the features listed above. Bits 
zero and one determine the number of bits per pixel. Bits two and three 
determine the pixel width in clock cycles. Bits four through six 
determine the first break in the video address and hence the display 
width in bytes. Bits seven through nine determine the second break in 
the video address and hence the display height in bytes. Bit ten turns 
the sync outputs into inputs which can reset the horizontal and vertical 
timers for rapid locking to an external video source. Bit 11 controls 
encrustation, which is the overlaying of an external video source using 
an external video multiplexer. The multiplexer is controlled by the 
"INC" pin of the A/V/CD controller/coprocessor. Selected bits of the 
colour are used to control encrustation. Bit 12 controls border 
encrustation, which is the same as bit 11 but only applied to border 
colours. Bit 13 sets a variable resolution mode. Bit 14 sets the colour 
hold mode, in which colour 0 is replaced by previous non-zero colour in 
current scan line. Bit 15 enables Pixel clock widths of three and five 
based on Bits 2, 3, and 15, as shown in the table below. 




28 



Bit 2 


Bit 3 


Bit 15 


Pixel Clock 


U 


u 


0 


Four clock cycles /Pixel 


1 


U 


1 


Two Clock Cycles/Pixel 


U 


1 


U 


One Clock Cycles/Pixel 


1 


1 


0 


Undef ined 


0 


0 


1 


Three Clock Cycles/Pixel 


1 


0 


1 


Five Clock Cycles/Pixel 


0 


1 


1 


Undefined 


1 


1 


1 


Undefined 



The video/memory controller 78 also has a pixel mask register and a 
palette index register. For every bit set in the mast register, the 

15 corresponding bit in the pixel is replaced by the bit from the index 

register. The upper bits in the index register form the high part of the 
palette address for 4 -bit pixels. The border colour register is a 16 -bit 
register that defines the border colour. The colour is displayed in the 
same way as 16 -bit pixels: bits zero to four are blue* bits five to ten 

20 are green, and bits 11 to 15 are red. 

The video/memory controller 78 also has two screen address 
registers that define the 24 -bit base address of the screen in system 
memory 45. This is the address of the top left pixel on the screen. 

25 

The video/memory controller 78 also has an auxiliary video mode 
register MODE2 that provides additional control over video and various 
test logic. Bit zero enables the light -pen registers so that the 
horizontal and vertical counters can be read. Bit two enables the video 

30 timer, bits three and four determine the refresh frequency with one 

indicating a refresh frequency of clock/128, two indicating a refresh 
frequency of clock/256, and three indicating a refresh frequency of 
clock/512. Most DRAMs require a refresh frequency of 64 KHz or above. 
The refresh controller waits until eight or more refresh cycles are 

35 required then requests the SYSTEM ' bus and does the required number of 

CAS before RAS cycles. when bit six is set, the video mode is double 
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buffered and can only change during blanking. The CPU 48 sets this bit 
for clean mode changes in split screen operation. Bit seven inverts the 
polarity of vertical sync. Bit eight inverts the polarity of horizontal 
sync and bit nine is not used. 

The palette is a 256 by 18 bit block of RAM at F10000H - F103FFH. 
Each entry contains six bits each of green, red, green and blue. Each 
entry extends across two words. The blue and green bits appear in the 
high word. The red bits appear in the low word. Bits two through seven 
of the high word are blue; bits 10 through 15 of the high are green and 
bits two through seven of the low word are red. To write to an entry in 
the palette, the CPU 48 must first write the red bits to the low word, 
then the green and blue bits to the high word. The CPU 48 should only 
write to the palette during border or blanking or speckles will appear on 
15 the video. 

The cache controller 64 holds instructions and data for the 
microprocessor 4 8 to improve system performance. Ik byte instruction and 
2K data cache with write back is supported. 
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Video/memory controller 78 supports six interrupt sources: video 
input interrupt, three analog interrupts, CD block decoder interrupt, and 
a DSP 61 interrupt. The analog interrupts allow simple analog - to - digital 
converters to be implemented. A monostable multi vibrator is implemented 
from a diode, a capacitor, and a potentiometer. The capacitor is 
discharged by vertical sync and begins charging at a rate dependent on 
the potentiometer setting. when the voltage on the capacitor reaches the 
threshold of the input to the video processor 39, an interrupt is 
generated. The processor can then read the vertical counter to get a 
measure of how quickly the capacitor charged, an hence the potentiometer 
setting . 

The video/memory controller 78 also has an interrupt enable 
register allowing all six interrupts to be independently enabled or 
disabled. writing a logical ONE to any bit in the interrupt acknowledge 
write register clears the corresponding interrupt. The interrupt read 
register reflects all pending interrupts. 

The video/memory controller 78 decodes the 16 megabyte address 
range of the 80376 CPU 4 8 into the following memory map: eight megabytes 
of DRAM0 (OH - 7FFFFFH), seven megabytes of DRAM1 (800000H - EFFFFFH) 64 
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kilobytes of ROMO (F000OOH - FOFFFFH) , 64 K of internal memory (F10000H - 
F1FFFFH) , and a 896 K block of ROM1 (F20000H - FFFFFFH) . The 64 
kilobytes of internal memory comprises palette RAM, blitter registers, 
and DSP registers and memory. The palette address range was stated 
above. The blitter registers extend from the range F10400H to F107FFH . 
The DSP memory extends from F10800H to F18000H. 



The on -board screen RAM and system RAM is 512K of DRAM. The on.- 
board DRAM comprising the screen/system RAM may be either 16 -bits or 32- 
bits wide. Suitable DRAM are the TCS14170BJ 256 kilobyte by 16 -bit 
memory chip, manufactured by Toshiba. The size of the DRAM is determined 
by the video processor 39 during reset but does not directly affect the 
CPU 48. Instead, it allows the video/memory controller 78 to operate 
more quickly leaving more bandwidth available to other bus master 
candidates. Certain display and blitter modes are only possible with 32- 
bit memory. Two banks of DRAM may be attached, as indicated above. If 
small amounts of DRAM are attached, then they will be repeated throughout 
the memory map shown above. 



20 The bootstrap ROM is always 16 bits wide. The bootstrap ROM 

comprises two 27C512 erasable programmable read-only memories, 
manufactured by numerous manufacturers, thereby giving 128K of bootstrap 
ROM. Following a reset, the one megabyte window from F20000H to FFFFFFH 
containing ROM and internal memory is repeated throughout the 16 megabyte 

25 address range. This allows for a variety of processors to boot with the 

video processor 39. The memory map above is adopted the first time with 
the memory type register is written to by the CPU 48. The video/memory 
controller 78 performs page mode cycles on the system memory 4 5 wherever 
possible. These are quicker than normal memory cycles and occur if 

30 successive reads and writes are within the same page. The video/memory 

controller 78 needs to know the number of columns in the DRAM, which is 
programmed in the memory type register. In the memory type register, bit 
0 and 1 determine the number of columns in the DRAM, with 0 indicating 
256 columns, 1 indicating 512, 2 indicating 1024, and 3 indicating 2048. 

35 

The video/memory controller 78 supports seven types of transfers: 
a normal DRAM cycle (4 clocks), a page mode DRAM cycle (two clocks), ROM 
cycles (6 clocks) , internal memory (2 clocks) , external I/O (6 clocks) , 
interrupt acknowledge (2 clocks), and internal I/O (2 clocks). The CPU 
40 48 will cycle in one more clock cycle than the actual transfer. Internal 

bus masters can cycle in the transfer time. 
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The video/memory controller 78 uses a crystal oscillator for a 
crystal that is the 2X (2 times speed) clock for the CPU 48 and is a 
multiple of the television chrominance (chroma) subcarrier. This crysta 
clock is buffered and output to the CPU 48. The same clock is put 
through a divide by two and this is output as the main system clock. 
This clock is input to the video processor 39 through a separate pin. 
The reason for outputting and inputting the clock is so that the relative 
skew between the CPU 2X clock and the main system clock, can be adjusted 
one way or the other by adding small delays to either path. The crystal 
frequency also is divided by a programmable divider which can divide the 
crystal frequency by a number between 1 and 15 and produce an output 
waveform with an even mark to space ratio. This is used as the 
television colour subcarrier. 
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The chroma divider register is a 4 -bit register that defines the 
ratio of the television colour subcarrier (chroma) to the 2X crystal 
frequency. It should be programmed as follows: chroma = 2X crystal 
frequency/chroma frequency - 1. 

The video/memory controller 78 also has a status register. if the 
status register bit 0 is set, the video timing should be set up for PAL 
(European television signal standard) . if bit 0 of the status register 
is clear, then the video timing should be set up for NTSC. If bit 1 of 
the status register has been set, then there has been a light-pen input 
in the current field. This bit is set by the light-pen and cleared by 
the vertical sync. 



The video/memory controller 78 can be put into a mode during reset 
after which it only responds to two-word wide I/O locations and 64 K 
30 memory locations. The actual location of the i/o locations is determined 

by a chip select input so the locations can be determined externally. 
This -peephole" mode allows the video processor 39 to occupy only small 
gaps in the I/O and address memory map of the system 30. 



The registers are 32-bits wide and must, therefore, be accessed as 
two 16-bit accesses. To address all the I/O registers within the video 
processor 39. the regular i/o address of the required register is first 
written to the lower word (all) low) then that register can be read or 
written at the upper word (a[l] high). To address all the memory inside 
and outside the video processor 39 the 64K window can be moved to any 64K 
ooundary in the 16m address space normally decoded by the video/memory 
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controller 78 by writing to the bank register. The bank register is an 
eight-bit register providing the eight most significant bits when 
addressing memory in peephole mode. For example, to access the palette, 
formerly at F10000H, the CPU 48 must write 0F1H to the bank register and 
then read and write at the bottom of the peephole location, determined by 
the external chip select. 

The blitter 72 is a graphical coprocessor whose purpose is to 
perform graphics creation and animation as fast as possible (limited by 
the memory bandwidth) . It executes commands written by the CPU 4 8 and 
the DSP 61 into memory. It can perform arbitrarily long sequences of 
graphics operations by reading new command sets from system memory 45. 
while it is performing graphics operations, the blitter 72 becomes a 
SYSTEM' bus master, and denies the CPU 48 any bus activity whatsoever. 
This is reasonable because the blitter 72 is being used to perform 
operations that the CPU 4 8 would otherwise have performed, and is 
therefore speeding up program operation. This also removes the need for 
any synchronous control programming for bl it ting operations and the need 
for any interrupt generation hardware in the blitter 72. However, to 
allow real time programming of either of the other two processors (the 
DSP 61 and the compact disc DMA) , the blitter 72 will suspend its 
operation and grant the SYSTEM' bus to the DSP 61 or the compact disc DMA 
channels if they require a DMA transfer. It will also suspend itself and 
give up the SYSTEM' bus to the CPU 48 if an interrupt occurs. During any 
of these transfers, the current operation is suspended but will restart 
when the interrupt signal becomes inactive or when the DSP 61 DMA access 
completes . 

The operation of the blitter 72 is best viewed as a simple program: 

read command from memory 
for n=0 to outer_count 

read parameters from memory 
for m=0 to inner_count 

if SRCEN then read source from memory 

if DSTEN then read destination from memory 

write destination to memory 

next m 

next n 



The commands and operands are written to memory by either the CPU 4 8 
the DSP 61. 



The blitter 72 has several registers in the video processor 39 i/o 
space: (1) two writable blitter program address registers, which share 
the same I/O address as two readable blitter destination registers, (2) a 
writable blitter command register, which shares the same I/O address as a 
first readable blitter source address register, (3) a writable blitter 
control register, which shares the same I/O address as a second readable 
blitter source address register, (4) a readable inner count register, (5) 
a first writable blitter diagnostics register, which shares the same I/O 
address as a readable blitter outer count register, (6) a second writable 
blitter diagnostics register, which shares the same I/O address as a 
readable blitter status register, and (7) a third writable blitter 
diagnostics register . 



The blitter 72 may be operated in a variety of modes to perform 
graphics and block move operations. The blitter 72 has an internal 
architecture divided into three largely separate blocks: the data path, 
the address generator, and the sequencer. The data path contains three 
data registers: the source data register, the destination data register, 
and the pattern data register. The data path also contains a versatile 
comparator to allow intelligent blitting operations, and a logic function 
unit (LFU) to generate the output data. 

The address generator contains three address registers: these are 
the program address register used to fetch blitter commands, and the 
source register and the destination address registers. It also contains 
an arithmetic logic unit (ALU) with an associated step register to update 
addresses, and a multiplexer to generate the output address. 

The sequencer acts in software terms as the program that the 
blitter 72 runs, with two loops (an inner loop and an outer loop) and a 
several procedures, as illustrated above with the short simple program. 
The program is fixed, although various parts of its operation are 
conditional upon flags in the blitter command register and the loop 
counts are also part of the command. 

The data path contains three data registers and two data 
manipulation blocks: the logic function unit, which can combine the 
contents of the data registers in a number of useful ways to produce the 
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output data, and the comparator, which can perform certain comparisons on 
the data to inhibit write operations, and optionally stop blitter 
operation . 

The data path can handle data of four sizes: 32 -bit, 16 -bit, 8- 
bit, and 4 -bit. Long words (32 -bits wide) are used when performing fast 
block moves and fills. Pixels (4-, 8-, or 16 -bits wide) may be 
manipulated using all the blitter modes, such as line-drawing, multiple 
plane operations, character painting, etc. 



The majority of the data path is 16 -bits wide, which is the maximum 
screen pixel size. However, the source data register is 32 -bits wide, 
and the top 16 -bits of the source data register are used to produce the 
top 16 -bits of the data written in 32 -bit mode, regardless of the mode of 

15 the logical function unit. Thus, there are two 16 -bit wide registers 

(pattern data and destination data) and one 32 -bit wide data register 
(source data register) . The source and destination data registers are 
loaded from the source and destination addresses in system memory 45 when 
the corresponding read cycles are enabled in the inner loop. However, 

20 all three data registers are loaded at the start of blitter operation 

with the pattern data, and this may be used as an additional source of 
data, either in producing the output data or in the comparator. For 
example, the data in the pattern data register could be a mask, a pattern 
for writing, or a reference value, for example. The pattern data is 

25 loaded into both words of the source data register. 

The logic function unit generates the output data, which is written 
to the destination in system memory 45. It can perform any logical 
combination of the source and destination register pixels. "Source data 

30 pixels" may be selected from either of the source data register or the 

data pattern data register. The LFU selects any of the four Boolean 
minterms (A & B, A & B, A & B, and A & B) of the two sets of input data 
from the data registers, and generates the logical OR of the two selected 
minterms. This allows any logical combination of input data; thus 16 

35 functional possibilities exist. 

In 32 -bit mode, the LFU will normally be set to produce source 
data, because it is only 16 -bits wide. The upper sixteen bits written 
during a long-word write are always derived from the top sixteen bits of 
40 the source register. 
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The comparator can perform a variety of comparisons on the data in 
the source, destination, and pattern data registers. if its comparison 
conditions are met, then it generates an inhibit signal. The inhibit 
signal is used to inhibit a write operation, and optionally, to stop the 
blitting operation. The comparator may also be used to provide a pixel 
Plane effect, to give transparent colours, for collision detection and 
system memory 45 search operations, and as an aid to character painting. 

A multiple plane operation is supported by assigning a plane number 
to every pixel. This mode is only applicable to 4 and 8-bit pixels. In 
8-bit pixel mode, two of the 8 bits (bits 6 &7) are used giving two or 
four planes; in 4-bit (nibble) pixel mode, one of the 4 bits (bit 3 a bit 
7 of the two-nibble byte) is used giving two planes. The comparator can 
produce an inhibit output if the plane number of the destination data is 
not equal to or greater than the plane number of the source data, or any 
combination of these. This means the data being written onto the screen 
can be masked by data already present in a different plane. 

The comparator can produce and inhibit output if the entire source 
pixel is equal to or not equal to the destination pixel. This may be 
used, for example, for searching system memory 45 for a particular value 
and, more importantly, for designating a colour to be transparent and 
holding the transparent colour value in a data register. This applies to 
16-, 8-, or 4-bit pixels. 

The blitter 72 also has a comparator bit to pixel expansion mode 
operation. This comparator operation allows bit to pixel expansion of 
data, used, for example, for character painting. in this mode, the 
comparator selects a bit of the source byte based on the value of the 
inner counter and inhibits the write operation if this bit is a logical 
ZERO. 
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The blitter 72 makes provision for handling three pixel resolution 
modes. These are: 16 -bit mode where each word corresponds to one pixel. 
8-bit mode, where each byte corresponds to one pixel, and 4-bit mode, 
where each byte corresponds to two pixels, m 8- and 16 -bit pixel modes 
the data path is handling one pixel at a time, and operation is straight 
forward. in 4 -bit pixel mode, however, only half of the byte that is 
read from or written to system memory is the current pixel, therefore, 
certain additional requirements are placed on the data path. m a 4 -bit 
mode write operation, unchanged destination data is written to the half 
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of the data byte that does not correspond to the current pixel. Thus, 
destination reads must always be enabled in 4 -bit mode (set control bit 
DSTEN) . This must be done because there is no provision for writing less 
than one byte into main memory. 

5 

It is also possible that the source 4 -bit pixel address and the 
destination 4 -bit pixel address point in different halves of the 
corresponding bytes of RAM. If this is the case, a shifter swaps the two 
halves of the source data. In 4-bit mode, the two nibbles (half a byte; 
10 4 bits) of the pattern byte should normally be set to the same value. 

Note that the pixel with program in the blitter 72 does not have to match 
the display width, and the most efficient way of moving large amounts of 
data is 32 -bit mode- Recall that such mode transfers must be long-word 
aligned and the system must be fitted with 32 -bit RAM. 

15 

The blitter 72 also has an address generator. The address 
generator contains three address registers, an increment or step 
register, an address adder, and an address output multiplexer. The three 
address registers hold the source address, the destination address, and 

20 the program address. Each of these registers is a 24 -bit register 

allowing the blitter 72 to address up to 16 megabytes. In addition, the 
source and destination address registers contain a nibble bit used in 
4 -bit pixel mode. The program address register holds the address that 
the program is fetched from, and is incremented by one word each time a 

25 memory cycle is performed using it. This register is always even, thus, 

bit 0 must always be a logical ZERO . 

The source and destination address registers are updated after each 
cycle, and at other times, using an adder that allows them considerable 

30 flexibility in the objects to which they refer. All source and 

destination address updates, may. be performed optionally on just the 
bottom 16 to 19 bits of the address register. This means that the 
blitter 72 will then effectively operate in 64K, 128K, 256K, or 512K 
pages. In this mode, if an address overflows within a page, it will wrap 

35 and the overflow or underflow will be lost. 

The blitter 72 also has an address adder, which is a 25 -bit wide 
adder used to update addresses. It allows either a constant value of .5, 
1, or 2 or a variable stored in one of the step registers, to be added to 
40 an address value. It can also subtract the same values. The 25th bit is 

the nibble part of the addresses, as stated above. An increment of one 
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pixel has a different effect on the address depending on the current 
setting of the screen resolution. 

All address registers are updated automatically at the end of the 
appropriate memory cycles; source read for the source of address 
register, and destination write for destination address register. 
Addresses can be made to wrap vertically by using the SWRAP and DWRAP 
bits in the blitter command, and horizontally by using the SLWRAP and 
DLWRAP bits in the blitter control register. 

The address output multiplexer provides the external address to the 
system memory 45. It provides three types of addresses: source address, 
destination address, and the program address. These are derived directly 
from the corresponding address registers. 

when the blitter 72 is drawing lines, the address registers are 
used in a different way than normal. The destination address register is 
used as the line draw address, and the source address register and the 
step register are used as delta one and delta two respectively. During 
line drawing delta two is subtracted from delta one, and the borrow 
output produced is used to determine what is added to the destination 
address register. For further details, see the section on line drawing 
below. 

The blitter 72 also has a sequencer which controls the operation of 
the blitter 72. The flow of control is best considered at two levels. 
There is an outer loop governing the overall flow of control and an inner 
loop which performs the actual blitting or line drawing operation. The 
three sections within the outer loop: the command read procedure, the 
parameter read procedure, and the inner loop. 



The inner loop performs the actual blitting or line drawing 
operations. An inner loop cycle can contain up to three memory cycles. 
These are a read from the source address, a read from the destination 
35 address, and a write to the destination address. All three cycles are 

optional. if the loop includes a source read, or a source read and a 
destination read, then the comparator inhibit mechanism is tested before 
the destination write occurs. This allows the write cycles to be 
bypassed when a comparator inhibit condition is met. when the comparator 
inhibit conditions are met, it is possible to have the current operations 
cease and control returned to the CPU 48. The program may then examine 
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the address registers to determine where the inhibit has occurred/ so 
that collision detection may be performed. The CPU 4 8 may then determine 
whether to resume the operation or abort it. The inner loop performs 
operations until the inner loop counter reaches zero. The inner loop 
counter is a 10 -bit counter, so the inner loop can iterate any number of 
times from 1 to 1024 . 

The blitter 72 makes provision for collision detection by allowing 
operation to stop when a comparator write inhibit occurs. when this 
happens, control returns to the CPU 48, which may then examine the 
internal state of the blitter 72 to determine what has caused the 
collision. At this point, the CPU 48 may choose to allow the blitter 72 
to resume the operation it was performing, or may reset it back to its 
idle state. Either a reset or a resume command must be issued before the 
blitter 72 may be used for another operation. Note that while the 
blitter 72 is in the suspended state, a new value may be written to the 
command register, so that the collision stop mechanism may be disabled. 

The parameter read procedure is a very straightforward sequence 
that loads a new set of parameters to the inner loop. It reads from 
memory, in order, the inner loop counter value, the step register values, 
and the pattern value, which is used to preset the data registers. The 
inner count effectively becomes the number of times the inner loop is 
executed. The step registers are used for address incrementing and the 
pattern register is used for data manipulation. 

The parameter read procedure is called as part of a command read 
procedure at the start of a blitting operation and is also called if 
required by a blitting operation, as determined by the PARRD control bit. 
Extra parameter reads occur between passes through the inner loop to 
allow parameters to be altered, thereby allowing operations such as 
irregular shape painting and run* length encoded data decompression. 

The command read procedure is used to start a new blitting 
operation. The blitter 72 starts in an inactive reset state, which 
represents the normal inactive state of the blitter 72. From this state 
a command register .write is performed to start the blitter 72, usually 
preceded by a write to the program address register. A full set of 
operational parameters is loaded from program count address which is 
auto - incremented, and control passes out of the command read loop. when 
a blitting operation is complete, a new command is read from tho program 



count address and if this command leaves the blitter 72 in run mode, ther 
a new set of parameters is loaded and another operation is started. 
Otherwise the blitter 72 enters its stopped state and returns the SYSTEM' 
bus to the CPU 48. The above mechanism allows the blitter 72 to perform 
arbitrarily long sequences of graphics commands without requiring any 
processor intervention. This is extremely useful because processor I/O 
write cycles are relatively slow in comparison to blitter memory reads. 

Normal operation of the outer loop starts on exit from the command 
read loop. The parameter read loop is then entered to read the first set 
of parameters and the inner loop is entered with the inner counter being 
loaded to its initial value before the start of operation. The outer 
counter is then decremented, and, if it is zero, the command read loop is 
entered. Then either or both of the source address and destination 
address registered may be updated with the contents of the step register. 
The parameter read loop may then be optionally entered to update various 
inner loop parameters, before the inner loop is entered again. The two 
loops allow the blitter 72 to perform operations on with two-dimensional 
screen structures, with the outer loop address register updates moving 
screen address pointers onto the start of the structure on the next line. 
The parameter read loop adds flexibility while allowing the screen 
structure parameters to be altered on a line-by-line basis. 

The blitter 72 also has a memory interface state machine, which 
controls the cycle timing generation and the bus arbitration of all 
memory cycles. The blitter 72 assumes control over the SYSTEM' bus from 
the CPU 4 8 for the duration of a blitter command sequence. This is 
subject to the bus handover latency discussed above, but as soon as the 
blitter 72 is granted the SYSTEM' bus its operation will start. 

The memory interface will give up the SYSTEM' bus to the DSP 61 or 
the compact disc read channel as soon as one of these requests the 
SYSTEM' bus, pausing only to complete any current memory cycle. 

Interrupts will also cause the blitter 72 to suspend operation, 
unless masked in the blitter control register. The blitter 72 detects 
the state of the. interrupt line itself and uses this to suspend 
operation. Operation will resume as soon as the interrupt line resumes 
to its prior state, which occurs when the CPU 48 write to the acknowledge 
port* occurs. This may not be necessarily the end of the interrupt 
service routine, therefore programmers should be wary of stack crawl, and 



should normally keep interrupts disabled during a service routine. The 
blitter 72 resumes operation as soon as the interrupt line is cleared 
without intervention from the CPU 48. The blitter 72 only responds to 
the internal interrupt sources (the video interrupt the analog input 
interrupts and compact disk interrupts) . Any external CPU interrupt 
source has no effect on the blitter 72. 

The blitter 72 has numerous modes of operation. The simplest 
operations performed by the blitter 72 are those involving copying one 
block of system memory 4 5 to another and filling a block of system memory 
45 with a predefined value. These operations can be performed on linear 
parts of system memory 45 and on arbitrary screen rectangles. The 
destination data register is used as the address of the system memory 45 
being modified and the source address register is used as the address of 
the data being copied, if it is a copy operation. 

When the operation is to be performed on linear areas of memory, 
most of the address control bits will be set to zero. The step register 
is not used, and the only requirement is to determine whether the copy 
will be made with the address incrementing or decrementing, in setting 
DSIGN and SSIGN appropriately. Note that the initial value placed in the 
address register should be the bottom of the area upon which the 
operation is to be performed if the sign bit is not set and at the top if 
it is set. In both cases, the first pixel read or written will be the 
first address. The length of the operation will be placed in the inner 
counter and the outer counter set to one. 

If the block being operated upon is very large both the inner loop 
and outer loop counters may have to be used and the number of pixels 
operating on will be given by the product of the inter and outer counter 
values. When either or both of the source and destination data are 
rectangles rather than linear areas, then the inner loop counter will 
contain the rectangle width and the outer loop counter the rectangle 
height . 

The appropriate step register is set to the address increment from 
the right-hand side of the rectangle around to the left-hand side on the 
next line. The SRCUP and DSTUP bits are set according to whether the 
source or destination are rectangles. In 8- or more bits per pixel mode, 
neither SRC EN nor DSTEN will be used for memory fill, bit SCRCEN should 
be set for memory copy. In 4 -bit pixel mode, DSTEN must always be set as 



41 



10 



well, so that a destination read is performed to avoid corrupting the 
other pixel. Note that using this method will be slower than otherwise. 

The blitter 72 draws lines based on the well known digital 
differential analyzer (DDA) algorithm. The basis of this algorithm is 
that for a given line one of the x address or the Y address is always 
incremented for every pixel drawn, while the other one is also 
incremented if a suitable arithmetic condition is met. The algorithm 
used by the blitter 72 computes the arithmetic condition that causes the 
conditional increment by repeated subtraction of the smaller of dx or dy 
from a working value with the larger being added back when underflow 
occurs, effectively using division to calculate the gradient. The 
notation -dx" refers to the distance along the X axis that the line 
corresponds to and is given by | (XI - X2) | where XI and X2 are the X 
15 coordinates of the 2 points and the vertical bar notation means the 

magnitude or absolute value of their difference. Thus if a line is being 
drawn from (XI. Yl) to (X2.Y2), then dx = | (XI - X2) | and dy = | (Yl - 
Y2)|. From these. Dl (referred to as "delta one" above) is given by the 
larger of dx and Dy, D2 (referred to as "delta two" above) by the 
smaller. Then, for each pixel drawn, D2 is subtracted from a working 
value which is initially set to Dl/2 and the sign of the result of this 
subtraction (indicating underflow) is the arithmetic condition for the 
conditional part of the screen address update. When this underflow 
occurs, the original value of Dl is added back to the working value. It 
can be seen that the ratio of dx to dy will give the frequency with which 
of this underflow and adding back occurs. The ratio between them is of 
course the gradient of the line. 
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The values used to create a line draw are set in the blitter 
command as follows: the starting point of the line is the destination 
address, Dl is placed in bits 10 to 19 of the source address register and 
Dl/2 is placed in bits 0 to 9 . Dl is also the inner counter value 
although Dl plus 1 should be used if both end points of the line are to 
be drawn. D2 is placed in the destination step register. if dx is 
3 5 greater than dy. then the YFRAC flag is set, otherwise it is cleared. 

SSIGN gives the sign of the X- address updates, DSIGN gives the sign of 
the Y- address updates. 
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While drawing lines, all the registers in the address section are 
occupied in computing the line address; t-hus the blitter has no ability 
to move data from somewhere else when drawing lines. Therefore, the data 
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written at the line address has to be given either directly by the 
pattern data or by combination of the pattern register and the data 
already there, according to the logical function unit. Consequently, 
SRCEN should not be set, otherwise the blitter would produce seemingly 
random data. While drawing lines the inner counter is set to the length 
of the line, and the outer counter is set to one. In 8 or more bits per 
pixel mode, DSTEN need not be set, unless used for read-modify- write 
operations. In 4 -bits per pixel mode, DSTEN must always be set so that a 
destination read is performed to avoid corrupting the other pixel. 



The blitter 72 also has the ability to paint characters on the 
screen in a single operation. Character painting as far as the blitter 
72 is concerned involves painting a rectangular area up to 8 pixels wide 
and of arbitrary height. The pixels in this area are either written to 
15 or left unchanged according to a bit pattern. This mode is not 

restricted to character painting, but may also be used to expand any 
graphics stored as a monochrome bit plane. 

During character paints, the source register addresses the bit 
2 0 pattern, normally part of the font, where each byte corresponds to one 

row of the character. Thus, blitter fonts may be up to 8 pixels wide 
however, wider fonts may be used, but these will require more than 1 
blitter paint operation to paint a character. Character painting is 
essentially a block move from the character font located in system memory 
25 45 to the destination address. 

The data is arranged with the bit corresponding to the left -most 
pixel in the least significant bit, and the top of a character at the 
lowest address. If the data is less than 8 pixels wide, then the least 
30 significant bits of the font data are not used. 

The destination address register is used to address the area of the 
screen to which the character is to be painted. Normally this area has 
been cleared to the required background colour by a previous blitter 

35 operation. The destination address is initialized to the top left-hand 

corner of the character. The character to be painted is a rectangle, 
and, therefore, the destination address is programmed correspondingly. 
The inner counter is sent to the width of the character and the outer 
counter to its height. The destination step register is set to the 

40 screen width less the width of the character. The DSTUP bit is used to 
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allow the destination address to be updated between passes through the 
inner loop. 

inner loop control bits DSTEN and SRCENF are set, character 
painting being the reason for the existence of SRCENF. This allows the 
font byte for each row to be read just once. The comparator is used to 
control the painting of pixels, therefore the CMPBIT control bit is set, 
to enable its bit to byte expansion mechanism. 



The colour to be painted is set as the pattern, and this will 
normally be held in the pattern data register. in 4 -bit pixel mode, 
DSTEN will be set, and the destination data register will hold the read 
values so that the other half of the byte may be written back 
undisturbed. The source data register holds the font pattern, as 
15 mentioned above. 
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The blitter rotate and scaling mode uses the shading ALU, but 
instead of producing three DDA-based data values, it produces two DDA- 
based address values, X and Y. Normally, these values are used to 
traverse a source data field at arbitrary angles and rates so that the 
destination data corresponds to a scaled and/or rotated version of them. 

The red value generator gives the X value and the green value 
generator gives the Y value. The blue value generator is not used, and 
clearly shading cannot be used in conjunction with this mode. As the 
rotation requires higher accuracy than shading, four extra integer bits 
are added to the X and Y values. These are set up in rotate registers 
zero and one. All calculations are performed to 10 point bit accuracy. 

As with shading, the delta values are added to X and Y after each 
pixel is drawn in the inner loop. The step values are added in the outer 
loop, and both the SRCUP and DSTUP flags must be set for them to be 
added. The delta and step values may be either positive or negative, and 
no add or saturation occurs, unlike shading mode. 

Normally, rotation and scaling are performed by setting the 
destination address pointer to performing normal raster scan over the 
destination rectangle, while the source pointer traverses over the source 
data at a suitable gradient and rate. This ensures that the destination 
data is contiguous, and that no more blits (blitter operations) than 
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necessary are required. The source data should be surrounded with a 
suitable transparent colour if the target area is not rectangular. 

A blitter command is given as a table of data in memory. The 
5 blitter 72 loads the contents of the table into its registers and 

performs the specified operation. The blitter 72 will receive successive 
sets of commands until a STOP instruction is read into the command 
register. 

10 The blitter program address must be set up before the command word 

is issued. The blitter program address is given by the program address 
registers, which together form the full 24 -bit address. The program must 
lie on a word boundary. 

15 A full table of blitter command data starts with a command word. 

However, the first blitter command in a sequence has its command word 
written to the command register by an I/O cycle of the CPU 48; thus, the 
blitter command starts reading the command data from the second word. 
Similarly, the last blitter command need consist of no more than a 

20 command word with the run bit clear. 

A blitter command takes the form of numerous command bits and 
control bits, a 24-bit source address, a 24-bit destination address, a 
10 -bit outer count value, a 10-bit inner count value, a 12 -bit signed 
25 source step, a 12 -bit signed destination step, and a 15 -bit pattern 

value. If the SHADE bit is set, then 9 additional words are fetched: 
red, green and blue initial values (6 integer bits and 10 fraction bits) , 
red, green and blue delta values (same) and red, green and blue step 
values (same) . 

30 

The command bits are as follows. Setting the RUN bit causes the 
blitter 72 to start operation. It is used when writing to the command 
register as an I/O port to start the blitter 72 reading a command. If 
the blitter 72 loads a command with the RUN bit cleared as part of a 

35 command read, then operation ceases. Setting the COLST bit causes 

operation to stop if a collision (write inhibit) occurs. From that 
point, print operation can be resumed by the CPU 48 or aborted, and 
various internal registers may be read. Setting the PARRD bit requires 
the blitter 72 to read a new parameter set from the program counter 

4 0 address, every time the inner loop exits and the outer, loop has not 

reached zero. Setting the SRCUP bit requires the contents of the step 
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register to be added to the source address on exit from the inner loop i 
the outer count has not reached zero. Setting the DSTUP bit requires th* 
contents of the step register to be added to the destination address on 
exit from the inner loop if the outer count has not reached zero. 
5 Setting the SRCEN bit enables the source address read in the inner loop. 

This also causes the source address register to be incremented according 
to the pixel size. Setting the DSTEN bit enables a destination address 
read in the inner loop. This does not affect the destination address 
register, which is incremented as part of the destination write cycle. 
10 Setting the SRCENF bit causes the source address to be read when the 

inner loop is first entered, but not subsequently entered. This is a 
special case of SRCEN and is relevant to the character paint mode, as 
described above. SRCENF has no affect if SRCEN is set. The two bits 
PSIZEO and PSIZE1 select the pixel size, 0 to 3 corresponding to 4, 8, 

15 16 / ^d 32 bits respectively. 32 bits is for data moves in a 32-bit 

system only, as described above. The 2 -bits WIDTHO and WIDTH1 select the 
screen width, in bytes, 0 to 3 corresponding to 256, 512, 1024, and 2048 
bytes, respectively. Setting LINDR puts the blitter 72 into line-drawinc 
mode. This mode uses both the source and destination address registers 

20 to generate the line -draw address, which may be used for both reading anc 

writing. Setting the YFRAC bit indicates to the blitter 72 which of the 
X and Y addresses have the fractional increment in line-drawing mode. It 
is set if the Y address has the fractional increment. Setting the PATSEL 
bit selects the pattern data register to replace the source data register 

25 as the source input to the logical function unit. This bit is relevant 

to character painting, where the source data register will contain the 
font data, and the pattern data register contains the ink colour. 
Setting the shade bit enables output from the shading ALU as write data. 
This bit is only valid for 8- and 16 -bit pixels. 

30 

The blitter 72 has several types of control bits: source control 
bits, destination control bits, logic function unit control bits, and 
comparator control bits. The blitter 72 has several source control bits. 
Setting the SWRAP bit causes source address updates to wrap on a 

35 programmable boundary, as opposed to running linearly through memory. 

Bits SWRAPO and SWRAP1 control the size of the SWRAP function, which 
makes the source address pointer wrap vertically, with 0 to 3 
corresponding to 64K, 128K, 256K, and 512K screens, respectively. 
Setting the SRCCMP bit selects the source data register as the source 

40 input to -the comparator. if it is cleared, the pattern data register is 

used. Setting the SLWRAP register makes the source pointer wrap within 
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the line width for inner loop updates. Setting the SSIGN bit sets the 
sign used when updating the source address. Setting it causes the source 
address to be decremented rather than incremented. This bit makes X 
negative in line-drawing. 

The blitter 72 also has several destination control bits. Setting 
the DWRAP bit causes destination address updates to wrap on a 
programmable boundary/ as opposed to running linearly through memory. 
Bits DWRapo and DWRAPl control the size of the dwrap function, which 
makes the source address pointer wrap vertically, with 0 to 3 
corresponding to 64K, 128K, 256K, and 512K screens, respectively. 
Setting the DSTCMP bit selects the source data register as the source 
input to the comparator. If it is cleared, the pattern data register is 
used. Setting the DLWRAP register makes the source pointer wrap within 
the line width for inner loop updates. Setting the DSIGN bit sets the 
sign used when updating the source address. Setting it causes the source 
address to be decremented rather than incremented. This bit makes Y 
negative in line-drawing. 

The blitter 72 also has logic function unit control bits. The 
logic function unit controls the data that is written in a destination 
write cycle. The LFU allows any logical combination of the source and 
destination data. This is achieved by each of the LFU bits LFUO through 
LFU3 selecting one of the minterms, with the output being given by the 
logical OR of the selected terms. A 0 value corresponds to NOT source 
and NOT destination, 1 corresponds to NOT source and destination, 2 
corresponds to source and NOT destination, and 3 corresponds to source 
and destination. There are, therefore, sixteen possibilities. 

The blitter 72 also has several comparator control bits. Setting 
CMPPLN enables plane mode where the three comparator functions operate on 
the plane number bits as opposed to the entire pixel. Setting the CMPEQ 
bit causes the comparator to inhibit an inner loop write, if in plane 
mode the priority of the destination pixel is equal to the plane priority 
of the source pixel, or if the entire pixel is the same if not in plane 
mode. Setting the CMPNE bit causes the comparator to inhibit an inner 
loop write, if in plane mode the priority of the destination pixel is not 
equal to the plane priority of the source pixel, or if the entire pixel 
is not the same if not in plane mode. Setting the CMPGT bit only 
operate? in plane mode, and causes the comparator to inhibit the write if 
the plane priority of the destination pixel is greater than the plane 
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priority of the source pixel. Setting the CMPBIT gives a bit to byte 
expansion scheme. it causes the comparator to generate an inhibit by 
selecting a bit of the source data register using an inner counter, and 
generating an inhibit if the bit selected is a zero. The selection is 
given by 8 in the inner counter selecting bit 0, 7 selecting bit 1, 6 bit 
2 , and so on . 
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The program address register points to the source of blitting 
operation commands. Data is read from it sequentially upwards through 
10 memory. it must always be even (i.e.. blitter operations must lie on 

word boundaries) . Register 0 corresponds to address bits 0 through 15 
and register 1 to address bits 16 through 23 and bits 0 through 7. 

Some of the above blitter registers are visible in the I/O space of 
15 tne cpu 48 • In addition, some blitter status and control bits are 

accessible to the CPU 48. As mentioned above, the blitter 72 has 7 word- 
wide read registers and 4 word-wide write registers. Any unused bits in 
the write register should be written with a 0. The I/O registers appear 
starting from I/O address 40H. These registers are also available in the 
memory map, principally so the DSP 61 can access them, starting at the 
same offsets as I/O, but at base address F10400H (i.e., subtract 40H and 
add F10400H to get the memory address) . The first blitter destination 
register corresponds to bits 0 through 15 of the destination address 
register. Bits 0 through 7 of the second blitter destination register 
correspond to bits 16 through 23 of the destination address register. 
And bit 15 of the second blitter destination register corresponds to the 
destination address nibble part of the destination address register. The 
first blitter source register corresponds to the bits 0 through 15 of the 
source address register. Bits 0 through 7 of the second blitter source 
register correspond to bits 16 through 23 of the source address register 
and bit 15 of the second blitter source register correspond to the source 
address nibble part. Bits 0 through 9 of the blitter inner counter 
correspond to the inner counter value. Bits 0 through 9 of the blitter 
outer counter correspond to the outer counter value. The blitter status 
register gives a variety of blitter status information. Bit 0 indicates 
that the comparator plane priority greater than condition is met. Bit 1 
indicates that the comparator plane priority equal condition is met. Bit 
2 indicates that the comparator plane priority not equal condition is 
met. Bit 3 indicates that the comparator pixel equal condition is met. 
Bit 4 indicates that the comparator pixel not equal condition is met. 
Bit 5 indicates that the comparator bit to pixel condition is met. Bit 
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*j corresponds to the run bit stating that the blitter is currently- 
active, or operation is suspended by a CPU interrupt or a collision stop- 
Bit 14 indicates that the blitter has stopped for a CPU interrupt. Bit 
15 indicates that the blitter has stopped because of a collision 
detection. The blitter program address register is loaded with bits 0 
through 15 of the blitter program address. Recall that bit 0 of the 
register is always 0 because blitter programs must lie on word 
boundaries. The second blitter program address register is loaded with 
bits 16 through 23 of the blitter program address in bits 0 through 7. 
The other 8 bits are 0. The blitter command register corresponds to word 
0 of the blitter command, and is used to set up the command when the 
blitter is started. Blitter DMA will then start from word one of the 
command . 

The blitter control register has three bits: bit 0 which is an 
interrupt stop mask masks interrupts from the blitter' s bus control unit 
when set, with a result that the blitter will not stop when an interrupt 
occurs, bit 1 causes the blitter to resume operation after a collision 
and is used to restart the blitter after a collision has been detected. 
Recall that a collision is detected when the COLST bit is set. The 
blitter will resume the operation which it has suspended. Note that it 
is possible to reprogram the blitter command register while the blitter 
is in the collision stop state, so the COLST bit among others may be 
changed, and bit 2 resets the blitter to a quiescent state after 
collision and is used to abort the operation the blitter was performing 
when a collision stop has occurred. Note that after a blitter collision 
stop occurs, either a resume or a reset should be issued to the blitter. 
The blitter 72 also has three rotate registers. Bits 0 through 3 
correspond to the top four bits of the integer part of the X address, the 
bottom six bits of the 10 -bit value are the integer part of the red 
value. Bits 4 through 7 correspond to the top four bits of the integer 
part of the X increment, the bottom six bits of this ten-bit value are 
the integer part are the red integer value. Bits 8 through 11 correspond 
to the top four bits of the integer part of the X step, the bottom six 
bits of this ten-bit value are the integer part are the red integer 
value. with the second rotate register bits 0 through 3 correspond to 
the top four bits of the integer part of the Y address, the bottom six 
bits of this ten-bit value are the integer part are the green integer 
value. Bits 4 through 7 correspond to the top four bits of the integer 
part of the Y increment, the bottom six bits of the ten -bit value are the 
integer part of the green integer value. Bits 8 through 11 correspond to 
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the top 4 bits of the integer part of the Y step, the bottom six bits of 
the ten-bit value are the integer part are the green integer value. in 
the third rotate register setting bit 0 causes the rotate address to 
replace the destination. Setting bit 1 causes the rotate address to 
replace the source address. Setting bit 2 sets rotation mode, as opposed 
to shading mode. And bits 10 through 15 correspond to the top bits of 
the rotate address . 

The DSP 61 audio coprocessor is a general purpose arithmetic 
coprocessor with sufficient power to implement a high performance music 
synthesizer. Synchronous serial outputs are provided for a generation of 
stereo audio signals with 16 bit precision, giving a sound quality 
normally associated with compact disc technology. The DSP 61 is micro - 
programmable from the host CPU 48 and the instruction set is sufficiently 
15 flexible to enable the user to program the device to fulfill many 

different functions that are quite different from that of "music 
synthesizer." Such applications might include algorithmic speech 
generation, audio analysis using fast Fourier transform techniques, and 
three-dimensional graphics rotations. The DSP 61 uses Harvard 
architecture (separate program and data buses) for maximum data 
throughput. The DSP 61 has an arithmetic logic unit (ALU) . 
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The ALU features a hardware 16 -bit by 16 -bit hardware 
multiply/accumulate as well as addition, subtraction, and logical 
25 functions. There is also a separate serial divide unit, which generates 

one quotient bit per tick. The carry bit from the adder/subtracter is 
stored in a separate latch and can be either used to propagate carry for 
multiple precision arithmetic operations or can be used for conditional 
instructions. All instructions may be made to be dependent on this bit 
being set. Data transfers within the device are all 16 bits wide, with 
the exception of internal transactions within the multiplier/accumulator. 
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The DSP 61 is a very simple, very fast processor intended primarily 
for sound synthesis, but also capable of other computational tasks as 
35 noted above. It executes all instructions in one processor cycle; these 

instructions are executed at the system clock speed (typically 20 to 33 
megahertz). During sound synthesis, the DSP 61 has its timing controlled 
by timers in an audio digital - to - analog converter (DAC) interface. These 
DACs are double-buffered, and if a DAC write is about to cause overflow, 
then operation is suspended until the buffer is empty. So :.ong as the 
software to executes loops at sample rate, and as long as -ne average 
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loop time is less than the sample period, then occasional loops can be up 
to twice as long. Because the loop may contain more instructions than 
will fit in the program RAM, the DSP 61 has an indexed addressing mode, 
which allows the same piece of code to act on several voices. 

5 

The DSP 61 is a Harvard Architecture device, thus the program RAM 
and the data RAM are separate, with cycles occurring in both RAM blocks 
at the same time. A one-cycle pipeline is used; therefore, during each 
clock cycle two events occur: an instruction is fetched, and the data 

10 transfer associated with the previous instruction takes place. This has 

the odd effect that an instruction after a jump is executed. The DSP 61 
has two arithmetic logic units (ALUs, not shown) : a typical ALU and a 
multiply/accumulate ALU; several registers: an x operand register, a 
second operand register, an AZ register, which holds the result from the 

15 ALU, and an MZ register, which holds the result from the 

multiply/accumulate register. The DSP 61 also has a DMA channel and a 
divider . 

Operation of the DSP 61 is fairly simple. in the first tick of an 
20 execution of an instruction, the opcode is read from the program RAM into 

the instruction decoder. In the second tick, while the next instruction 
is read from the program RAM, a data transfer is performed either from 
system memory 45 to a register or a register to system memory 45, as per 
the first instruction. 

25 

The ALU within the DSP 61 is a 16-bit arithmetic logic unit, with 
the same functions as a Texas Instruments 74181, which is well known in 
the art. Common arithmetic operations are encoded as instructions; 
uncommon instructions may be performed by directly setting up the ALU 
30 mode bits with the general purpose arithmetic instruction (GAD . 



The DSP 61 also has a multiplier/accumulator, which is a second ALU 
to perform 16 by 16 signed/unsigned multiplies to yield a 32 bit result. 
In addition to this, it may also perform multiply/accumulate operations, 

35 where the product of the multiply is added to the previous result. A 

result is accumulated to 36 bits to allow for overflow. Multiplier 
operations actually take two ticks, although the multiplying instruction 
itself completes in one tick. This means that the instruction following 
a multiply or a multiply accumulate may not involve the MZ register or 

40 the X register. 
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The DSP 61 also has a divider. The division unit appears as a set 
of registers in the internal DSP 61 space. It is capable of unsigned 
division on 16- or 32 -bit operands, and produces a quotient and a 
remainder. 

The DSP 61 also has a DMA channel. The DMA channel appears as a 
set of registers in the DSP 61 data memory space. These are two address 
registers and a data register, a DMA transfer is initiated by writing an 
address to the first of the two address registers, dma transfers have a 
latency period, which must be allowed to elapse before performing further 
DMA. The DMA state machine is responsible for requesting the SYSTEM' 
bus, and when it is granted, performing the transfer, after which the 
SYSTEM' bus is released. 



15 



20 



25 



30 



35 



40 



In the alternative, a word may be written to the second of the two 
address registers with a hold bit set. This will request the SYSTEM' bus 
and retain it until the hold bit is cleared. Such a DMA transfer may be 
efficient when performing successive multiple transfers, but is generally 
less efficient for single transfers because the DSP 61 program cannot 
determine when the SYSTEM' bus is granted, and therefore has to wait the 
maximum possible latency. DSP 61 memory is generally visible in both the 
DSP's internal data address base and in the host address base. 

The DSP 61 has a DSP memory 76 associated with it. The DSP memory 
76 comprises program RAM, data RAM. a register/constant table, and a sine 
ROM (all not shown) . The DSP memory 76 in general is accessible in both 
the DSP's internal address space as well as the address space of the 
system memory 45. The DSP program RAM is 512 18 -bit words. These 
locations may only be written by the CPU 48, and are program read-only as 
far as the DSP 61 i s concerned. Program RAM does not appear in the DSP 
internal address space. The program ram is not accessible to the host 
when the DSP 61 is running. Each DSP instruction has a 7-bit opcode and 
an 11 -bit address vector. AH microcoded instructions (with the 
exception of multiply or multiply/accumulate operations) are completed in 
185 nanosecond cycle. All instructions are system memory 45 to register 
transfers or register to register transfers; immediate values are not 
allowed. Thus, if a constant is needed for a given instruction, it is 
not available in the constant table, a data RAM location must be set 
aside for the value. The DSP 61 also allows conditional instructions and 
indexed addressing, if bit 12 of the instruction code is set, then the 
instruction is executed only lf - h . carry bit in the ALU is also set if 
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bit 11 in the instruction code is set, then the 9 -bit address vector in 
the instruction code is added to the 9 -bit value in the index register to 
produce the address and data memory operated on by the instruction. The 
extra two bits are programmed by loading the values into an extra bits 
register then writing the word into the desired location. 

The DSP 61 has numerous move commands, which move data from and to 
memory and registers. Several other commands are available, including 
adding, subtracting, ANDing, ORing, adding with carry, a NOP, the GAI 
described above, and an INTRUDE command, which allows the DSP memory 76 
to be accessed by the CPU 48. 

The sine ROM is 256 16-bit words of full sine wave two's complement 
sine wave values. 

The data RAM is 512 16 -bit words. 

Data may be transferred between the CPU 48 and the DSP 61 either 
under control of the DSP 61 or under the control of the host CPU 48. 

The DMA transfer mechanism is based upon the DSP 61 becoming the 
bus master on the SYSTEM' bus and accessing the system memory 45. The 
DSP 61 is one of the highest priority bus masters, and will therefore be 
granted the SYSTEM' bus by the current bus master as soon as the current 
bus master is able to give up the SYSTEM' bus. The worst case for giving 
up the SYSTEM' bus is the situation where the CPU 48 is the bus master, 
because the 80376 or 80386SX processor can take a considerable amount of 
time to release the SYSTEM ' bus. DMA transfers are started by a write to 
the first DMA address register, as stated above. Transfer of status 
information and the high part of the address should already have been 
written to the second DMA address register; similarly write data should 
already have been written to the DMA data register in the case of write 
transfers. when a transfer is initiated, the DSP 61 requests the SYSTEM' 
bus and when the SYSTEM ' bus is granted to the DSP 61, the DSP 61 
performs the transfer and then releases the SYSTEM' bus. Completion of 
this operation may be polled or the programmer may choose to allow the 
maximum possible latency to elapse before using read data and/or 
initiating another transfer. 

A second bus acquisition technique may be performed which uses the 
hoxd bit in the second of the two DMA address registers to request the 
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SYSTEM' bus. This may be more efficient if the DSP 61 wishes to perform 
multiple transfers consecutively, because the SYSTEM' bus is not releasee 
between transfers. The hold bit in the second DMA address register must 
be cleared before the DSP 61 will release the SYSTEM ' bus. This 
mechanism is generally not recommended because the DSP 61 will have 
control of the SYSTEM' bus for significant periods of time without any 
activity, which is wasteful of overall memory bus bandwidth and could 
potentially disturb CD DMA transfers. If using the second technique, the 
DSP 61 must first request the SYSTEM' bus before performing any DMA 
transfer. it has no means of detecting that it has gained the SYSTEM' 
bus, and must therefore wait the maximum number of bus instructions. 
Once the DSP 61 has acquired ownership of the SYSTEM' bus it may then 
proceed to perform bus cycles. It may perform an arbitrary sequence of 
read and/or write cycles and should relinquish control of the SYSTEM' bus 
15 at the end of these. 

Data transfer may also be performed between the CPU 48 and the DSP 
61 under host CPU 48 control. All the internal memory of the DSP 61 is 
mapped into the host address space, when the DSP 61 is in stop mode, the 
host may write program memory locations just as if they were in normal 
system memory 45. when the DSP 61 is running, however, the program 
memory is not available to the host. DSP 61 data memory is only 
available by the INTRUDE mechanism. To ensure that DSP 61 operations are 
not disturbed in any way, data transactions can only take place in the 
data when the DSP 61 is executing INTRUDE instructions. When the DSP 61 
is stopped, it may be considered to be effectively executing intrude 
instructions constantly. 



20 



25 



30 



35 



40 



CPU 48 to DSP program RAM 76 transfers may be performed using the 
blitter 72 only while the DSP 61 is not running. Likewise, the blitter 
72 cannot access DSP data RAM while the DSP 61 is executing. in short, 
both the blitter 72 and the CPU 48 may modify DSP program RAM 76 only 
while the DSP 61 executes an intrude instruction. 

The DSP 61 can cause the blitter 72 to perform very fast block 
moves of DSP code from system RAM to DSP program RAM. Thus, the DSP 61 
and blitter 72 can team up to effectively provide the DSP 61 with more 
program RAM than is actually available. 

The DSP 61 also has a serial audio digital-to-analog convertor 
(DAC) interface. The serial DAC interface allows the DSP 61 to both 
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drive a synchronous serial (l a s or similar) DAC, and to input data from a 
synchronous serial data source such as a CD drive. The interface timing 
can be internally generated if no input device is attached, but if a data 
source is present, then it must be used to determine the timing. An 
5 internal overflow detector prevents the DSP 61 from writing to the DAC 

before the previous output data has been fully output. This is governed 
by write to the first of two DAC registers. Therefore, DAC transfers 
should take the form: write to the first DAC register, write to the 
second DAC register, read input values. These should be performed in 

10 close succession (less than 16 instructions) . There is no detection of 

underflow, and should this occur, then the previous output value will be 
output again. The DAC values are doubled buffered, so that although 
audio code should loop at an average rate less than or equal to the 
sample period, it is possible for occasional passes through the loop to 

15 take up to two sample periods. This may be useful for exception 

processing . 

The DSP 61 contains an arithmetic logic unit (ALU) compatible with 
the Texas Instruments 74181 device. 

20 

The video processor 39 also has a compact disc DMA controller 54. 
This CD controller contains the following functional blocks: A simple 
synchronous serial interface (for I*S and similar) , a CD ROM block 
decoder and a DMA channel. The mechanism allows a serial data stream to 
25 be transferred to system memory 45, either directly, or by first passing 

through a block decoder. This allows an external block decoder to be 
used, in case of problems or incapabilities in the internal one. An 
interrupt can be generated when a transfer completes, given the transfer 
length counter reaching zero. 

30 

The compact disc controller synchronous serial interface supports 
the Philips data format, which is well known in the art. The Philips 
data format has a clock, a word select line and a data line. The word 
select leads the data by one tick of the clock, and the data is aligned 

35 against the most significant bit (MSB) of a 32 -bit datum. A low on the 

word select line indicates left data and a high on the word select line 
indicates- right data. The synchronous serial interface also supports the 
Philips block decoder output formula. The bit ordering is reversed, and 
the first bit is aligned against the first bit of the datum. The word 

40 select format can be either the Philips data format, the Sony data 

format, or the Matsushita data format. 
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The CD drive controller 54 also has a block decoder. The block 
decoder synchronizes to the start of the 2352-byte sectors, performs the 
descrambling, and computes the EDC (error detection code) to detect 
errors. It operates in either short mode where a 2048 data bytes are 
transferred after the header, or a long mode, where the 2340 bytes after 
the sync pattern are transferred. This allows the header and error 
correction data to be read if desired. The header size is either 
programmable to either 4 or 12 bytes to support CD drive mode 1 and 
CDI/XA mode 2 form 1, but header matching is only performed on the main 
4 -byte header. The mode 2 forms are only supported by operating in long 
mode and extracting the required data. Header matching is performed on 
the first sector of the transfer to ensure that the correct data is being 
read. The desired header value should be programmed into the header 
registers. If a multisector transfer is performed, then no further 
15 matching occurs after the first sector. 
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Typically, normal transfers are performed in short mode, with the 
long mode being used when an error has been detected, so that the 
operating software can attempt to correct it. Multiple sector transfers 
are supported by giving a count of the total number of long words to be 
transferred. Errors will abort multiple sector transfers. Errors can 
take the following forms: unreliable data, EDC error, and no sync. It 
is possible to poll the decoder to determine its current status. The CD 
drive controller 54 also has a DMA interface. The DMA interface can 
transfer to 2 16 -bit words at a time into system memory 45. It can take 
either the output from the internal block decoder or the output from the 
synchronous serial interface. It has an address counter that runs 
upwards through system memory 45. The DMA interface has a transfer 
length counter for direct transfer from the serial interface. In 
"forever mode" the DMA address register wraps within a 32 kilobyte 
buffer, and a counter is ignored. This may be useful for CD audio data, 
or for real time and data handling such as "full -motion video" 
decompression. A CPU 4 8 interrupt is generated every time the address 
pointer wraps around the buffer. 

Similarly, the present invention contemplates that many of the 
characteristics heretofore offered in set top devices used as accessories 
to television receivers may be incorporated directly into devices which 
are here called intelligent television receivers. One such intelligent 
television receiver is illustrated in Figure 6 and identified there by 
reference character 10'. The circuitry described above with reference to 
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Figures 3 through 5 will be incorporated within the housing or cabinet 
11' of the intelligent receiver 10', in order that the receiver may 
respond to and cooperate with a remote control 20 as herein described. 
Inasmuch as such circuitry has been described in detail hereinabove, such 
5 description will not here be repeated. 

Similarly, the present invention contemplates that the benefits of 
these inventions may be gained through use of personal computer systems. 
One such personal computer system is illustrated in Figure 7. 

10 

Referring now more particularly to Figures 7 through 9 of the 
accompanying drawings, a personal computer system embodying the present 
invention is there shown and generally indicated at 90 (Figure 7) . The 
computer 9 0 may have an associated monitor 91, keyboard 9 2 and printer or 

15 plotter 94. The monitor 91 functions as the display device in displaying 

visual images to a human observer, in similarity to the CRT 12, 12' of 
the television receivers illustrated in Figures 1 and 6. The computer 90 
has a cover 9 6 which cooperates with a chassis 9 8 in defining an 
enclosed, shielded volume for receiving electrically powered data 

20 processing and storage components for processing and storing digital 

data, as shown in Figure 8. At least certain of these components are 
mounted on a multilayer planar 100 or motherboard which is mounted on the 
chassis 98 and provides a means for electrically interconnecting the 
components of the computer 9 0 including those identified above and such 

25 other associated elements as floppy disk drives, various forms of direct 

access storage devices, accessory cards or boards, and the like. 

The chassis 9 8 has a base and a rear panel (Figure 8) and defines 
at least one open bay for receiving a data storage device such as a disk 

30 drive for magnetic or optical disks, a tape backup drive, or the like. 

In the illustrated form, an upper bay 101 is adapted to receive 
peripheral drives of a first size (such as those known as 3.5 inch 
drives) . A floppy disk drive, a removable media direct access storage 
device capable of receiving a diskette inserted thereinto and using the 

35 diskette to receive, store and deliver data as is generally known, may be 

provided in the upper bay 101. 

Prior to relating the above structure to the present invention, a 
summary of the operation in general of the personal computer system 90 
40 may merit review. Referring to Figure 9, there is shown a block diagram 

of a personal computer system illustrating the various components of the 
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computer system such as the system 90 in accordance with the present 
invention, including components mounted on the planar 100 and the 
connection of the planar to the I/O slots and other hardware of the 
personal computer system. Connected to the planar is the system 
5 processor 102. While any appropriate microprocessor can be used as the 

CPU 102, one suitable microprocessor is the 80386 which is sold by INTEL. 
The CPU 102 is connected by a high speed CPU local bus 104 to a bus 
interface control unit 105, to volatile random access memory (RAM) 106 
here shown as Single Inline Memory Modules (SIMMs) and to BIOS ROM 108 in 
10 which is stored instructions for basic input/output operations to the CPU 

102. The BIOS ROM 108 includes the BIOS that is used to interface 
between the I/O devices and the operating system of the microprocessor 
102. Instructions stored in ROM 108 can be copied into RAM 106 to 
decrease the execution time of BIOS. 

15 

While the present invention is described hereinafter with 
particular reference to the system block diagram of Figure 9, it is to be 
understood at the outset of the description which follows that it is 
contemplated that the apparatus and methods in accordance with the 
20 present invention may be used with other hardware configurations of the 

planar board. For example, the system processor could be an Intel 80376 
or 80486 microprocessor. 

Returning now to Figure 9, the CPU local bus 104 (comprising data, 
address and control components) also provides for the connection of the 
25 microprocessor 102 with a math coprocessor 109 and a Small Computer 

Systems Interface (SCSI) controller 110. The SCSI controller 110 may, as 
is known to persons skilled in the arts of computer design and operation, 
be connected or connectable with Read Only Memory (ROM) 111, ram 112, and 
suitable external devices of a variety of types as facilitated by the I/O 
connection indicated to the right in the Figure. The SCSI controller 110 
functions as a storage controller in controlling storage memory devices 
such as fixed or removable media electromagnetic storage devices (also 
known as hard and floppy disk drives), electro-optical, tape and other 
storage devices. 
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The bus interface controller (bic) 105 couples the CPU local bus 
104 with an I/O bus 114. By means of the bus 114, the BIC 105 is coupled 
with an optional feature bus such as an Industry Standard Architecture 
(ISA), MICRO CHANNEL, EISA, PCI , or other bus having a plurality of I/O 
slots for receiving adapter* cards 115 which may be further connected to 
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an I/O device or memory (not shown) . The I/O bus 114 includes address, 
data, and control components. 

Coupled along the I/O bus 114 are a variety of I/O components such 
as a video signal processor 116 which is associated with video RAM (VRAM) 
for storing graphic information (indicated at 118) and for storing image 
information (indicated at 119). Video signals exchanged with the 
processor 116 may be passed through a Digital to Analog Converter (DAC) 
120 to a monitor or other display device. Provision is also made for 
connecting the VSP 116 directly with what is here referred to as a 
natural image input/output, which may take the form of a video 
recorder/player, camera, etc. The VSP may take the form of the video 
processor 3 9 and associated circuitry described above with reference to 
Figures 3 through 5, in which event the CPU 102 may function, as to video 
control, similarly to the CPU 48 described above. 

The I/O bus 114 is also coupled with a Digital Signal Processor 
(DSP) 121 which has associated instruction RAM 122 and data RAM 124 
available to store software instructions for the processing of signals by 
the DSP 121 and data involved in such processing. The DSP 121 provides 
for processing of audio inputs and outputs by the provision of an audio 
controller 125, and for handling of other signals by provision of an 
analog interface controller 126. 

Lastly, the I/O bus 114 is coupled with a input/output controller 
128 with associated Electrical Erasable Programmable Read Only Memory 
(EE PROM) 129 by which inputs and outputs are exchanged with conventional 
peripherals including floppy disk drives, a printer 94, keyboard 92, a 
mouse or pointing device including a remote control such as the device 
20, and by means of a serial port. In the form illustrated in the 
Figures here under discussion, the pointing device is in the form of a 
mouse 13 0 joined to the computer system by an elongate flexible conductor 
131. 

In all instances, a personal computer system practising these 
inventions will have a remote control device. In such systems, "remote" 
control is characterized by the remote control device being usable at 
some distance of separation from an associated video display device. 
That distance of separation may be quite small, on the scale of inches, 
or comparatively larger, on the scale of feet, meters, yards or more, as 
will become more clear from the description which follows. 
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As discussed hereinabove, the remote control device may be a three 
ax ls device or, in some instances, a two axis device. The two devices 
are distinct and noninterchangeable, as will become clear from the 
discussion which follows, a "three axis" device, as the terminology is 
here used, is one in which a human user may manipulate a control element 
m three dimensions to effect the generation of control signals which 
will be effective to direct modification of visual images displayed on 
the associated video display device. A -two axis- device permits 
manipulation in only two dimensions. 

A three axis device permits a type of operation which has been 
described as "press to select". That is, manipulation of the control 
element may direct movement of a selection display element, such as a 
cursor, from side- to-side and up-and-down through the field of a 
displayed visual image and then be used to also make a selection of a 
display feature. Side-to-side movement may, for example, be coupled to 
thumb pressure to one side or the other; up-and-down, to pressure away 
from or toward the manipulator. Selection, then would be coupled to 
thumb pressure along the third axis, as by pressing downwardly against 
the control element. An early example may be found in the disclosure of 
Garrett United states Patent 5,065,146, issued 12 November 1991 and held 
xn common with the inventions here described. 

A two axis device, in contrast, typically provides a separate 
control element for selection. Examples may be found in a conventional 
personal computer pointing device such as those known as a mouse, and in 
conventional remote controls as typically provided with such consumer 
electronic devices as television receivers, video cassette recorders 
audio amplifiers, compact disk players, video disc players, and the like, 
in both instances, one control element (a rolling ball in the case of the 
mouse and up-down or right-left rocker switches or stepping keys in the 
case of the conventional remote control) may direct movement of a 
selection display element, suc h as a highlighted band, across a displayed 
video image, and then a second control element (in the case of a mouse, a 
button; a conventional remote control, a push switch, is manipulated to 
may a selection of a display feature. 

The two types of remote control devices are other than 
interchangeable . 



40 
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Either of the two types of remote control devices is contemplated 
by these inventions as being capable of transmitting command signals 
coordinated in a predetermined manner to manipulation of the control 
element (s) by the human observer. Either may be coupled to the display 
controller to transmit at a frequency which is outside direct sensing by 
the human observer in a variety of ways, including radiation of light at 
frequencies not visible to a human observer (infrared or ultraviolet), or 
radio frequencies. Either may be tethered, or coupled by a flexible • 
conductor as is commonly the case with a personal computer pointing 
device. 

The control element (s) manipulable by a human user are contemplated 
as being in a variety of forms. One, described hereinabove, may be known 
as a "wiggle stick", and takes the form of a elongate element standing 
upright for engagement with a user's thumb. Another may be a "wobble 
plate", a somewhat flat and planar or shallowly dished member, typically 
round in configuration, mounted to pivot relatively freely about a 
central point, typically maintained in a neutral position by a resilient 
bias such as a spring, and which can be depressed by a user's thumb in 
any one of at least four selected directions. Yet another may be a 
trackball, which is somewhat similar to an inverted personal computer 
system mouse, in that a freely rotatable element or ball is provided and 
supported in a manner that enables signals to be generated indicating 
rotation of the ball in its mount. And still yet another may be an 
inertial or "air" mouse. Such a device typically has an inertial 
platform and sensors capable of generating signals indicating 
displacement of the mouse in space. 

This invention contemplates that control programs (including 
operating system and application programs) will be stored in the system 
RAM 45 or flash ROM 49 and executed in the display controller. In 
accordance with this invention, such control programs make use of a 
particularly compact programming language now to be described. The 
language and programming to be described are particularly useful in 
limiting the amount of memory which must be provided in a consumer 
product, where the costs of such memory is an important factor in selling 
price. However, it is to be understood that less compact programming 
languages, and thus control programs, may be useful where the expense of 
additional memory capability (up to and including inclusion of a hardfile 
or fixed disk direct access storage device) can be borne. 
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The computing system disclosed herein is "open", i.e. a system tha 
will run future applications which are not currently defined. The syster 
has limited storage for applications. It is therefore important to 
minimize the size of the applications that run on the system, so they car 
reside in a small amount of storage. 

Two tasks may inflate the size of application software. One is tht 
User interface (herein identified as UI) . This part of the software 
drives hardware which interacts with the user, such as a display, a 
speaker, a keyboard, or a mouse. The other is the External interface 
(herein identified as EI), the system's access to information from its 
own storage and information from other systems. This part of the 
software drives hardware such as a disk drive or a modem. 

15 The first step in reducing the size of application program data is 

to remove from the applications the software for the user interface and 
the external interface. The present invention contemplates that the 
software for these two tasks is built into the system, not the 
applications. Referring to Figure 10: 
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Box 1 The user interface (UI) hardware (display, keyboard, etc.) is shown 
at the top of the figure. 

Box 2 The UI Engine is permanent software in system ROM which handles the 
user interface. For instance, the UI Engine (box 2) would display 
information on the screen (part of box 1) at the request of an 
application (in box 3). Sound, keyboard activity, and other user 
input/output would be handled by the UI Engine. Note the arrows 
indicating that the UI Engine interacts with both the UI hardware 
and the applications. One objective achieved by this invention is 
to make the ui Engine small so that it requires less system ROM. 



Box 3 The box in the middle of the figure drawn with a dotted outline 

indicates application program data storage. The applications are 
35 stored in system RAM (readable and writable), so that an 

application can be added, removed, or modified. One objective 
achieved by this invention is to make the applications small so 
that more applications can fit in a given storage space. In Figure 
10, the size of the applications is potentially reduced because the 
user interface and the information interface are handled outside of 
the applications. 
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Box 4 The EI Routines are also permanent software in system ROM ; these 
routines handle the external interface hardware. For example, an 
EI Routine would dial a phone number on the modem at the request of 
an application. (Some of the EI Routines are in RAM instead of 
ROM, so that additional hardware support can be added in the 
future . ) 

Box 5 The external interface (EI) hardware (disk drive, modem, etc.) is 
shown at the bottom of the figure. 



This arrangement finds some parallels in other computer systems. 
For example, the DOS operating system makes file input/output functions 
global to all applications; these are EI Routines. The Microsoft windows 
environment provides a common UI for all applications. However, the 
15 primary goal of these systems is to make applications conform to 

standards, not to save space. 

The present invention provides a user interface that is based on 
"levels". At a given level, the user views information and makes a 
20 selection. The selection may cause a new level to be created below the 

current level, or the selection may cause the current level to be 
destroyed, returning to a previous level. 

In a preferred embodiment, each level in the user interface is 

25 represented by a graphic and text display similar to a paper index card 

drawn on the screen. The pieces of paper ("menu cards") are cascaded on 
the screen, as illustrated in Figure 12. As there illustrated, the user 
is currently at Level 2. Moving to level 2 has involved the sequence 
illustrated by Figures 10, 11 and 12. From Figure 10, the display of a 

30 full motion video image as received from a video/audio stream source such 

as broadcast television, a user may cause a first level of menu to appear 
in overlay over the video stream image by actuation of the selection 
feature provided on the remote control 20. Thereafter, the user may 
manipulate the cursor or pointer to be positioned over an indicated item, 

3 5 such as item 1 for "Weather", and again actuate the selection feature of 

the remote control 20. Thereupon the user interface will respond by 
creating the next level, Level 2, as illustrated in Figure 12. A return 
to the video/audio stream image alone can be effected by positioning the 
cursor/pointer in the field of the image and actuating the selection 

40 feature or stepwise by first returning to menu level 1 by positioning the 
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cursor over the heading "Information Highway" and actuating the selection 
feature of the remote control. 

This is a simple example of the user interface; a typical situatior 
is much more complex. For instance, some menu cards only present 
information, and do not allow any action except exiting to the previous 
level after viewing the information, other menu cards allow the 
selection of a combination of items. The user interface supports these 
and other types of menus. Any menu can have more than one page (as 
indicated by the "turn the page" symbol at the lower right hand corner of 
the card illustrated in Figure 12); turning a page does not imply moving 
to a different level. 

in a UI Engine in accordance with this invention and based on 
levels, each level is either a menu object or a flow object. These two 
types of objects are called "cards- in a preferred embodiment: 

1. A -menu card" implements a UI level. For instance, each of the two 
"pieces of paper" in Figure 12 is a menu card. 

2. A "flow card" implements a routine in a programming language. This 
type of card does not appear on a display screen as a UI level, and is 
hence invisible to the user. while a menu card presents a list of 
actions to the user, a flow card processes a list of actions with a "flow 
of control" determined by branches, loops, etc. 

Each of these cards can launch a card of either type, a menu can 
launch another menu or a flow in response to a user selection, a flow 
can launch another flow or a menu. Furthermore, each type of card can 
invoke an El Routine when it needs to use the external interface 
hardware. Each card can also invoke another UI Engine application; this 
transition is seamless to the user since the sequence of levels is not 
interrupted. 

To illustrate these points, consider an example from the preferred 
embodiment of the language. Figure 11 shows the beginning of a sample 

application . 

Each line in the language consists of two parts: 
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1. A "description", the text before the encircled A or "at" symbol 



2. An "action" , the text beginning with the encircled A ( n @ n ) . 

5 

Each card begins with a title line, such as "Level 1 ©Cardl menu". The 
description part of this line is the title of the card; the action is the 
card's label. Each card ends with a line containing only "@ n . Hence, 
three cards are shown in the example in Figure 12. 

10 

The application begins by displaying Cardl. The title of the Card 
is "Level 1" and the selectable items are "Selection A" and "Selection 
B". If "Selection A" is selected, it creates Card2, since this is the 
action in the "Selection A" line. Card2 is a flow card which immediately 
15 displays Card3 since the condition "1 — 1" is true. 



Figure 12 shows the three cards. Card2 is a flow card which is 
invisible to the user. Only Cardl and Card3 are displayed. 

20 Though the two card types are similar, there are some important 

differences. The following table shows the symmetry between menus and 
flows. 



Menu 


Flow 


Menu cards are visible to the user. 


Flow cards are invisible to the 
user . 


The user looks at the descriptions 
and selects an action . 


The system looks at the descriptions 
and selects an action 


If a line has no "®" , then the 
entire line is taken as a 
description and there is no action. 
This is because an action with no 
description is meaningless in a 
menu. 


If a line has no "@" , then the 
entire line is taken as an action 
and there is no description. This 
is because a description with no 
action is meaningless in a flow. 


Descriptions determine how the text 
appears on the screen: its position, 
colour, etc. 


Descriptions determine the flow of 
control in a programming language 
with branches, loops, etc. 
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When allocating resources such as variables and file streams, it i. 
useful to designate each resource with a level. The resource is 
automatically deallocated when its level is destroyed. For example, when 
a variable is created in the language, it is assigned a level. The 
variable is global to all levels, i.e. a card can send information to 
another card by putting it in a variable. However, the variable is 
destroyed when the level inside the ui Engine goes below the level of the 
variable. In other words, when the card where the variable was created 
is destroyed, the variable is destroyed with it. The variable -goes out 
of scope" when its level is destroyed. This is how the language handles 
all resources that can be allocated to cards. 

The present invention provides solutions to three problems. First, 
the size of the applications is dramatically reduced. Observe from the 
example in Figure 11 that the sample application is stripped down to bare 
essentials. Most of the content of the application is text rather than 
programming. The text can be compressed to less than half its original 
size by using standard compression techniques. The size of the 
programming can be reduced by compilation; however, this will not be 
necessary if a compression algorithm used for the text is adapted to also 
compress the programming. The resulting application approaches the 
theoretical minimum size, which is the size of its compressed text. 
Next, the size of the UI Engine is reduced. Observe that menu cards and 
flow cards have identical syntax. Both types of cards are made up of 
lines that have descriptions and actions. The actions which a menu can 
perform are the same as the actions which a flow can perform. (The only 
exception is that a flow allows actions which jump around in the flow, 
while these actions are meaningless in a menu.) variable resolution and 
other parsing operations are the same for both types of cards. Thus, the 
same software in the ui Engine processes both menu cards and flow cards. 
Last, the UI Engine running this language can be ported to any operating 
environment that is based on menu levels. This is because the language 
limits user input and output to a hierarchy of levels. The same 
applications could run in a variety of different environments. 

Menu systems are commonly implemented with the "menu" type of 
objects. For example, the Microsoft windows Software Development Kit 
includes a "Dialog Box Editor" which constructs the levels of the Windows 
menus. An object at each level can create an object at the next level, 
similar to one menu cheating another!' 
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Also, programming languages commonly use the "flow" type of object. 
For instance, objects in the C++ language can create instances of other 
objects, similar to one flow creating another. 

5 The uniqueness of the programming language here described is the 

way it mixes the two types of objects in a single unified language. 

Turning now to the range of menu construction and display 
capabilities envisioned for the systems described to this point in the 

10 present specification, it is contemplated that the command processor 

circuitry described hereinabove respond to manipulation of the remote 
control 2 0 by enabling the human observer to move a cursor image 
displayed over a video image to a menu item and select for execution a 
menu item overlain by the cursor image. Further, in a manner similar to 

15 the "drag and drop" functionality of certain personal computer system 

software, the command processor circuitry responds to manipulation of 
said remote control device by enabling the human observer to move a 
cursor image to a menu item, select for displacement a menu item overlain 
by the cursor image, and move a selected menu item across the visual 

20 image displayed by the visual display device. To aid in a user 

distinguishing the differing characteristics of portions of the display 
field, the cursor image signal may change the visual characteristic of 
the displayed cursor as manipulation of the input devices causes the 
displayed cursor to be moved to different areas of displayed visual 

25 images, as by making the cursor larger when over certain fields of the 

display. 

As illustrated in Figures 13 through 15, the display controller 
modifies displayed visual images by displaying over a portion of a live 

30 video images a menu display from which the human observer may select 

further modifications of said visual images. Thus, in Figure 13, a major 
portion of the available field is occupied by the video stream image (the 
image of a weather map, partially obscured in Figures 14 and 15 by the 
overlain menus and informational text) while a minor portion is occupied 

35 by the displayed menu(s). The menus offered may include, as in Figure 

14, a pull down menu display in which possible further modifications of 
the visual images and/or accessible information displays are displayed as 
tiled windows or as overlain windows or as cascaded windows. Certain of 
the accessible information choices, such as item 4 "Pizza", preferably 

40 provide access to remote services such as ordering take out food by means 

of the back channel communication such as a modem incorporated in the 
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system. Others, such as item 1 "Weather" will access information 
available from a data service such as local weather observations (as 
illustrated in Figure 15) . still others, such as item 5 -TV Guide- may 
lead to the selection of programming for viewing. 

The display controller may also modify displayed visual images by 
displaying the video stream image as a minor portion of the available 
field. Access to such a display is illustrated by the sequence of 
Figures 16 through 16. As indicated in the right hand portion of Figure 
16 (where menu selection elements are display as overlain onto a video 
stream image) , the menu display may mimic functional controls provided a- 
remote control functions in prior television receivers or video cassette 
recorder/players. Selection of the remote control functions enables use 
of the embodied icons such as the iconic representations of -channel up" 
or-channel down- found in remote control functionality for navigation 
among programming choices. However, by selecting "List" or -Menu", other 
services may be accessed. From the screen of Figure 16, selecting -Menu" 
wm take an observer to the screen of Figure 17, where the video stream 
images is displayed in a minor portion of the available screen area (the 
upper right hand corner) almost as if it were a so-called -picture in 
Picture", with the remainder of the viewing field being occupied by 
listing of available choices, a list of desired viewing options can be 
created by selecting the function "List" to add the currently viewed 
signal stream, then progressively selecting and adding other choices to 
the list. when viewed channels or signal sources are thus constructed 
into a list, the list may be named (such as "Fri Night- for favourite 
programs viewed that evening or -Kids" for programming specifically 
selected by or for children) and saved in system memory. Thereafter, the 
previously viewed and assembled list may be recalled for ready "channel 
surfing" among the preselected range of programming. when creation of a 
list is completed, the function "Done" may be selected to end the 
process . 
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CLAIMS 



1. A video processor which is operative to receive first and second 
analog video signals and produce an analog video output comprising: 

first means for converting said second analog video signal to a 
digital video signal and reducing the size of said digital video signal 
by a predetermined amount, 

a system memory coupled to said first means, for storing said 
reduced digital video signal, 

second means for producing synchronization information to 
15 synchronize said reduced digital video signal with said first analog 

video signal, 

encoder means for receiving said synchronization information and 
converting said reduced digital video signal to a third analog composite 
20 video signal, 

third means coupled to said encoder means and said second means for 
combining said first analog video signal with said third analog composite 
video signal to produce said output analog video signal comprised of a 
25 portion of said first analog video signal and a portion of said third 

analog video signal. 

2. The video processor of claim 1, wherein said output analog video 
signal comprises a full screen image on a display device of the first 

30 analog video signal and a picture- in - picture (PIP) image on said display 

device of the third analog video signal, said PIP image overlaying said 
full screen image. 

3. The video processor of claim 1, wherein said first means 
35 comprises: 

a video decoder for converting said second analog video signal to 
said digital video signal and converting said digital video signal to YUV 
colour space, and 

40 
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a scaler coupled to said video decoder for reducing the size of 
said digital video signal by said predetermined amount and placing the 
reduced digital video signal into said system memory. 

5 4. The video processor of claim 3, wherein said predetermined amount 

is in the range of between 1/9 and 1/4 of a full screen image. 

5. The video processor of claim 1, wherein said second means 
comprises : 

10 

a sync extract circuit for receiving said first analog video 
signal, providing said first analog video signal on one output, 
extracting first synchronization information from said first analog video 
signal and providing said first synchronization information on a second 
15 output, 

a video controller coupled to said encoder means for rasterizing 
said reduced digital video signal stored in the system memory and 
producing colour information. 

20 

6. The video processor of claim 5, wherein said second means further 
includes a genlock and timing circuit coupled to said sync extract 
circuit and said video controller for receiving said first 
synchronization information and providing second synchronization 

25 information to said encoder means and clock information to said video 

controller . 

7. The video processor of claim 5, wherein said encoder means is a 
multi standard encoder for receiving said second synchronization 

3 0 information from said genlock and timing circuit and said colour 

information from said video controller and producing said third analog 
video signal. 

8. The video processor of claim 1, further including a first VBI 
35 decoder coupled to said system memory for receiving said first analog 

video signal and extracting digital data from the vertical blanking 
interval of said first analog video signal and storing said digital data 
in said system memory. 

40 9. The video processor of claim 8, further including a second VBI 

decoder coupled to said system memory for receiving said second analog 
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video signal and extracting digital data from the vertical blanking 
interval of said second analog video signal and storing said digital data 
in said system memory, 

10. The video processor of claim 8, wherein said digital data 
represents electronic program guide data. 

11. The video processor of claim 8, wherein said digital data 
represents channel violence ratings. 

12. The video processor of claim 8, wherein said digital data 
represents closed captioning data for the hearing impaired. 

13. A video processor which is operative to receive first and second 
15 analog video signals and produce an analog video output comprising: 

a video decoder for converting said second analog video signal to a 
digital video signal, 

20 a scaler coupled to said video decoder for reducing the size of 

said digital video signal by a predetermined amount and placing the 
reduced digital video signal into a system memory, 

said system memory coupled to said scalar for storing said reduced 
25 digital video signal, 

a synchronizer for producing synchronization information to 
synchronize said reduced digital video signal with said first analog 
video signal, 

30 nal stored in the system memory and producing colour information. 

an encoder for receiving said synchronization information and 
converting said reduced digital video signal to a third analog composite 
video signal, 

3 5 a video multiplexer coupled to said encoder and said second means 

for combining said first analog video signal with said third analog 
composite video signal to produce said output analog video signal 
comprised of a portion of said first analog video signal and a portion of 
said third analog video signal. 

40 
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14. The video processor of claim 13, wherein said output analog video 
signal comprises a full screen image on a display device of the first 
analog video signal and a picture - in-picture (PIP) image on said display 
device of the third analog video signal, said PIP image overlaying said 
full screen image . 

15. The video processor of claim 13, wherein said predetermined amount 
is in the range of between 1/9 and 1/4 of a full screen image. 

16. The video processor of claim 13, wherein said synchronizer 
comprises : 



a sync extract circuit for receiving said first analog video 
signal, providing said first analog video signal on one output, 
15 extracting first synchronization information from said first analog videc 

signal and providing said first synchronization information on a second 
output , 



a video controller coupled to said encoder means for rasterizing 
said reduced digital video signal stored in the system memory and 
producing colour information. 



17. The video processor of claim 16, wherein said synchronizer further 
includes a genlock and timing circuit coupled to said sync extract 
25 circuit and said video controller for receiving said first 

synchronization information and providing second synchronization 
information to said encoder and clock information to said video 
controller . 

30 18 * The video processor of claim 16, wherein said encoder is a multi 

standard encoder for receiving said second synchronization information 
from said genlock and timing circuit and said colour information from 
said video controller and producing said third analog video signal. 

35 19 • The video processor of claim 1, further including a first VBI 

decoder coupled to said system memory for receiving said first analog 
video signal and extracting digital data from the vertical blanking 
interval of said first analog video signal and storing said digital data 
in said system memory. 

40 
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20. The video processor of claim 19, further including a second vbi 
decoder coupled to said system memory for receiving said second analog 
video signal and extracting digital data from the vertical blanking 
interval of said second analog video signal and storing said digital data 
in said system memory. 

21. The video processor of claim 19, wherein said digital data 
represents electronic program guide data. 

22. The video processor of claim 19, wherein said digital data 
represents closed captioning data for the hearing impaired. 

23. A method for processing a first and a second analog video signal 
with a video processor to produce an analog composite video output 
comprising the steps of : 

converting said second analog video signal to a digital video 
signal and reducing the size of said digital video signal by a 
predetermined amount, 

storing said reduced digital video signal in a system memory, 
producing synchronization information for synchronizing said 
reduced digital video signal with said first analog video signal, 

receiving said synchronization information and converting said 
reduced digital video signal to a third analog composite video signal, 

combining said first analog video signal with said third analog 
composite video signal to produce said output analog video signal 
comprised of a portion of said first analog video signal and a portion of 
said third analog video signal. 

24. The method of claim 23, wherein said combining step includes 
generating a full screen image on a display device of the first analog 
video signal and a picture - in • picture (PIP) image on said display device 
of the third analog video signal, said PIP image overlaying said full 
screen image. 

25. The method of claim 23, wherein said converting step includes the 
steps of : 
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converting said second analog video signal to said digital video 
signal and converting said digital video signal to YUV colour space, and 

reducing the size of said digital video signal by said 
predetermined amount and placing the reduced digital video signal into 
said system memory. 

26. The method of claim 25, wherein said predetermined amount is in the 
range of between 1/9 and 1/4 of a full screen image. 

27. The method of claim 23, wherein said producing synchronization 
information step includes the steps of: 

receiving said first analog video signal at a sync extract circuit 
15 and providing said first analog video signal on one output, 

extracting, with said sync extract circuit, first synchronization 
information from said first analog video signal and providing said first 
synchronization information on a second output of said sync extract 
20 circuit, 

rasterizing, with a video controller, said reduced digital video 
signal stored in the system memory and producing colour information. 

25 28 - The method of claim 27, wherein said combining step includes 

combining said first analog video signal output of said sync extract: 
circuit with said third analog composite video signal to produce a full 
screen image on a display device of the first analog video signal and a 
picture- in-picture (PIP) image on said display device of the third analog 

30 video signal, said PIP image overlaying said full screen image. 

29. The method of claim 27, wherein said system includes an encoder for 
performing said receiving the synchronization information step and said 
step of producing synchronization information step further includes; 



35 



receiving, with a genlock and timing circuit coupled to said sync 
extract circuit and said video controller, said first synchronization 
information and providing second synchronization information to said 
encoder means and clock information to said video controller. 



40 
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30. The method of claim 27, wherein said receiving said synchronization 
information step includes receiving, at said multi - standard encoder, said 
second synchronization information and said colour information and 
producing said third analog video signal. 

5 

31. The method of claim 23, further including the step of: 



receiving, with a first VBI decoder coupled to said system memory, 
said first analog video signal and extracting digital data from the 
10 vertical blanking interval of said first analog video signal and storing 

said digital in said system memory. 

32. The method of claim 31, further including the step of: 

15 receiving, with a second vbi decoder coupled to said system memory, 

said second analog video signal and extracting digital data from the 
vertical blanking interval of said second analog video signal and storing 
said digital data in said system memory. 



20 33. A video display system, substantially as hereinbefore described, 

with reference to figures 1 to 5 of the accompanying drawings. 

34. A combination, substantially as hereinbefore described with 
reference to figure 6 of the accompanying drawings. 

25 

35. A video display system, substantially as hereinbefore described, 
with reference to figures 7 to 9 of the accompanying drawings. 
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